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EXECUTIVE SUMMARY 



INTRODUCTION 

The National Library of Medicine's basic mission is to assist the progress of 
medicine and public health by improving the dissemination of biomedical 
information. In support of this goal, NLM developed MEDLARS, the automated 
Medical Literature Analysis and Retrieval System, and was a pioneer in offering 
online retrieval services over nationwide telecommunications networks. For the 
first 20 years of the system's existence, the principal users of MEDLARS were 
medical librarians who acted as search intermediaries for researchers and 
practitioners who needed to locate pertinent information in the health sciences 
literature. As more and more of these health professionals began to have access to 
personal computers, NLM took several steps to encourage them to search NLM's 
databases directly, including the provision of special short training courses in online 
searching and the development of GRATEFUL MED, a user-friendly interface to 
the NLM system. In the past three years the number of individual health 
professionals searching MEDLINE directly on the NLM system has increased 
dramatically. It is continuing to grow at the rate of over 200 users per month. 

In order to provide a retrieval service that individual health professionals can use 
easily and effectively, NLM needs to obtain more detailed information about this 
growing and important end-user population, their information needs, and their level 
of satisfaction with MEDLINE as available on the NLM system. This survey was an 
initial attempt to determine the demographic characteristics of individual users of 
MEDLINE, their methods of access to the system, their reasons for searching, and 
their level of satisfaction with current system features and capabilities. The 
information gathered will be used to identify system improvements that can provide 
a better level of information service to U.S. health professionals. 



METHODOLOGY 

An NLM study team developed a questionnaire to elicit the types of information 
desired with assistance from Market Dynamics, Inc., a contractor hired to assist in 
tabulating the survey results. After a pre-test involving a small number of users, the 
questionnaire was revised and submitted for review by the U.S. Office of 
Management and Budget as required by the Paperwork Reduction Act. The final 
version of the questionnaire appears as Attachment 1. 

The survey population included all who were presumed to be individual "end-users" 
of MEDLINE on the NLM system as of the end of July 1987. Questionnaires were 
mailed to these 4,311 individuals on October 2, 1987. There were two follow-up 
mailings, and returns were accepted until December 10, 1987. 

The gross return rate was 70.1% (2,970 responses); the usable return rate was 68% 
(2,716 responses). Since the entire universe of individual MEDLINE users was 
surveyed, sampling error does not affect the results. Because the return rate was 
high, non-response bias is negligible. We believe the survey findings to be 
representative of all individual users of MEDLINE as of July 1987. 
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DEMOGRAPHIC CHARACTERISTICS OF SURVEY RESPONDENTS 



Professions 

Nearly two-thirds (65.5%) of the respondents (2,710 individuals, 99.8% of the entire 
respondent group) identified themselves as physicians and more than a quarter 
(27.7%) as scientists. 286 (10.5%) said they were both physicians and scientists. 
More than 40 different medical specialties were listed by the 2,197 respondents 
(80.9% of the entire respondent group) who provided specialty information. 

Education 

Most respondents hold either MX). (1,760) or Ph.D. (570) degrees. The majority of 
the respondents (2,667 individuals, 98.2% of the entire respondent group) received 
their highest degree within the last 18 years: 40.1% during the 1970's and 27.1% in 
the 1980*s. 

Primary Work Place 

Nearly all of the individuals surveyed (2,701 individuals, 99.4% of the entire 
respondent group) indicated a primary work place. Almost half of the respondents 
(46\8%) work in an academic setting (college, university, or medical school). The 
remaining respondents are split rather evenly among private/solo practice (12.6%), 
group practice (123%) and hospital/clinic settings (14.6%). Only 5.4% work in a 
government agency and 33% in a private company or business. 



GENERAL SEARCHING BEHAVIOR 
Frequency of Searching 

More than two-thirds (68.6%) of the respondents (2,686 individuals, 98.9% of the 
entire respondent group) do all their own searches. Half of these perform two or 
fewer searches in an average month. For the respondents who perform at least one 
search in the typical month, the average number of searches performed is 4.3 
searches per month. 

About one-third of the respondents have someone else search for them at least 
occasionally. These respondents request an average of 2.5 searches from others 
each month. 

Searches Performed by Others 

A substantial majority (81.9%) of the searches respondents request from others are 
done by librarian/information specialists. 42.4% of the entire respondent group 
(1,151 individuals) indicated the reasons they occasionally have someone else search 
for them. The most frequently indicated reasons were: lack of time to search 
personally (593% of those who gave any reasons); the need for different expertise 
or system knowledge (38.7%); and lack of satisfaction with their own results for a 
particular search (27.5%). 70.6% of those who have others search for them (1,121 
individuals, 413% of the entire respondent group) expressed satisfaction with' the 
results, 8.9% are dissatisfied and 20.5% are neutral. 
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Searches Performed by Self 

Of the 2,171 individuals (79.9% of the entire respondent group) who indicated one 
or more factors that influenced them to do their own searches, 83.1% selected 
greater familiarity with the subject matter, 82.4% marked the ability to get results 
faster, and 65.5% enjoyment of searching. 

Level of Experience/Pattern of Use 

Less than 10% of the respondents (2,561 individuals, 94.3% of the entire respondent 
group) regard themselves as very experienced users of online databases. The large 
majority consider themselves to be somewhat experienced or not very experienced. 
Less than 8% indicated that they were not at all experienced. 

Of the 2,509 respondents (92.4% of the entire respondent group) who indicated how 
lone they had been searcning MEDLINE with their own codes, nearly two-thirds 
had had their codes for a year or less at the time of the survey. 27% had had their 
codes for one to two years. Only 10% had had their codes for more than 2 years. 

A substantial majority (82%) of the respondents (2,661 individuals, 98.0% of the 
entire respondent group) do not share their code with anyone else. 

More than half of the respondents (2,545 individuals, 93.7% of the entire 
respondent group) said their level of usage of the system had remained relatively 
constant since they received their own access codes. 30.0% indicated increased use 
and 123% said their use had decreased. 

Cost Considerations 

67.6% of the 2^54 respondents (94.0% of the entire respondent group) who 
provided an answer to the question on cost considerations indicated that cost rarely 
(26.6%) or never (41.0%) keeps them from performing a search. 23.1% said that 
costs considerations occasionally keep them from searching. Only 9.2% reported 
that cost considerations frequently keep them from searching. Cost is even less a 
consideration for those in group practice, private/solo practice, and private business 
than for respondents as a whole. 



CHARACTERISTICS OF MEDLINE SEARCHES 
General Search Purpose 

Of the 2^56 individuals (94.1% of the respondent group) who provided an answer, 
81.7% indicated that they typically search MEDLINE to satisfy an immediate 
information need; 10% typically search for information to stay current in their field. 
Only 5% typically search to learn about new areas. 

Numbers of Citations Desired/Retrieved 

Overall, more than half of the respondents (2,550 individuals, 93.9% of the entire 
respondent group) typically wish to retrieve all relevant citations from a particular 
time period rather than just a few relevant citations. The results vary substantially 
according to the respondents' work place and their most common search purpose, 
however. 65.2% of those who work in academic settings and 67.1% of those who 
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commonly use search information for research/testing purposes are interested in 
retrieving all relevant citations as opposed to 47.6% of those in group practice and 
49.8% oi those who most commonly use search information for patient care. 

More than half of the respondents (2,510 individuals, 92.4% of the entire 
respondent group) typically retrieve about the right number of citations. Overall, 
15% of the respondents indicated that they typically retrieve too few citations and 
23.6% indicated that they typically retrieve too many. A higher percentage (23.6%) 
of those who always search using GRATEFUL MED form screens retrieve too few 
citations compared to individuals who always search using command language 
(10.2%). 

2^99 individuals (88.3% of the entire respondent group) indicated the percentage 
of retrieved citations that they typically judge to be relevant to their inquiry. 58.6% 
of these respondents said that typically fewer than half of the citations they retrieve 
are relevant. 

Use of Information Retrieved 

Over 90% of the respondents (2^52 individuals, 94.0% of the entire respondent 
group) typically search for a subject rather than for an author or a journal title. 
Respondents were asked to rank the three primary areas in which they use 
MEDLINE search information. Most frequently mentioned were: research/testing 
(by 71% of respondents! patient care (69.1%), and education (66.0%). Overall, 
45.8% ranked research/testing as their single most common use; 32.8%, patient 
care; and only 12.4%, education. The results varied substantially by work place. 
68.8% of those in private /solo practice and 75.8% of those in group practice most 
commonly use MEDLINE for patient care. In contrast 66.9% of individuals in 
academic settings most commonly use MEDLINE for research/testing, purposes. 

Use ofMeSH 

About two-thirds of the respondents (2,544 individuals, 93.7% of the entire 
respondent group) always use (24.1%) or usually use (43.6%) MeSH terms when 
searching for specific subjects. Similar percentages of respondents find MeSH terms 
very useful or useful. 

Length of Searches 

For almost two-thirds of the respondents (2,533 individuals, 933% of the entire 
respondent group) a tvpical search takes less than 10 minutes. Another quarter 
takes 10-15 minutes. Nearly 90% of respondents think the length of time to conduct 
a search is reasonable. 



OPINIONS ABOUT THE SYSTEM 
Level of Satisfaction 

1,318 individuals (48.5% of the entire respondent group) indicated a level of 
satisfaction with the searches they perform for themselves. Of these, 72.5% are 
satisfied with their searches. 
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93.5% of the entire respondent group (2,539 individuals) rated their level of overall 
satisfaction with MEDLINE on the NLM system. 81.4% of this group were 
satisfied, and only 4.1% dissatisfied. The rest were neutral. 

Desirable Additions/Improvements to MEDLINE 

60.8% of respondents (1,977 individuals, 72.8% of the entire respondent group) 
would find no additional types of information valuable. Of those who would like to 
see additional information, 79.1% indicated the full-text of articles, 37.0% author's 
address, 31.4% research design, and 23.6% dosage information. 

78.8% of the entire respondent group (2,139 individuals) indicated features they 
would like to see added to MEDLINE by selecting from a list provided in the 
questionnaire. 71.8% of these selected improved MEDLINE backfile searching and 
69.2% improved printing capabilities. These two capabilities were also the single 
most desired features, by 35.9% and 27.9% of respondents respectively. 



ACCESS AND TRAINING 
Method of Access 

96% of respondents (2,708 individuals, 99.7% of the entire respondent group) have 
access to a microcomputer in their work places, and a similar percentage use a 
microcomputer to search MEDLINE. 

68.9% of the entire respondent group (1,870 individuals) always use a single method 
to access MEDLINE. Of the 2,474 individuals (91.1% of the entire respondent 
group) who provided information on access methods, 43.2% always use the 
command language, 26.9% always use the GRATEFUL MED form screens, 4.2% 
always use another front-end package, and 1.8% the GRATEFUL MED direct 
option. The remainder use more than one method to access MEDLINE. Higher 
percentages of physicians and those in private/solo and group practice always use 
GRATEFUL MED form screens in comparison to the respondent group as a whole. 
The percentage of those who always use command language increases as age 
(indicated by year highest degree received) decreases. The reverse is true for use of 
GRATEFUL MED form screens. 

Of the 2^92 individuals (88.1% of the entire respondent group) who answered the 
question regarding access problems, 53.8% reported none. The most frequent 
problem encountered was busy telecommunications lines (19.5%). 

Learning to Search MEDLINE 

2,203 individuals (81.1% of the entire respondent group) checked the methods they 
used to learn to search, with 2,059 individuals (75.8% of the entire respondent 
group) also indicating the single most helpful method. The frequently mentioned 
methods were: NLM sponsored training course (49.7%); GRATEFUL MED 
(45.3%); and self-taught (45.0%). The most helpful methods were NLM-sponsored 
training (42.7%) and GRATEFUL MED (31.3%). Of the 1,368 individuals who 
said they took either NLM's 3-5 day training class or the 6-hour Basics of Searching 
MEDLINE Course, over 83% were satisfied with the training, less than 4% were 
dissatisfied, and the rest were neutral. 
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COMMENTS ON THE SYSTEM 



Most Satisfactory Aspects 

747 individuals (27.5% of the entire respondent group) provided comments on their 
perceptions of the most satisfactory aspects of MEDLINE. 32.7% of these people 
mentioned the content of MEDLINE; 25.7% hours and availability; 24.6% speed 
and efficiency; 19.9% GRATEFUL MED; and 18.7% cost. The results varied based 
on access method. Higher percentages of those who always use the command 
language mentioned the content of the database and cost as compared to 
GRATEFUL MED form screen users. GRATEFUL MED itself was the most 
frequently mentioned satisfactory aspect by GRATEFUL MED form screen users. 

Least Satisfactory Aspects 

874 individuals (32.2% of the entire respondent group) provided comments on 
aspects of MEDLINE that they considered to be least satisfactory. Most frequently 
mentioned were GRATEFUL MED limitations, by 17.2% of those who provided 
comments; backfile searching, by 14.0%; and MeSH vocabulary/ indexing, by 
13.7%. Again the results varied based on access method. 



CONCLUSIONS 

The survey data reveal two different groups of individual users. The first group is 
concentrated in academic settings, uses MEDLINE primarily in support of research, 
and is typically interested in comprehensive retrieval. Most of the people in this 
group are physicians or physician scientists. . In July 1987, the majority in this group 
used the command language. Hie second group is made up predominantly of 
physicians who work in various clinical practice settings. These individuals use " 
MEDLINE primarily in support of patient care and are more likely than the 
"research'' users to wish to retrieve just a few relevant citations. In July 1987, a larger 
percentage of this group already used GRATEFUL MED as compared to the 
"research" group. At the time the survey was conducted, the "research" group was 
slightly larger than the "patient care" group; but there is evidence that the "patient 
care" group is growing at a faster pace. 

Although the "research" and "patient care" groups have some key differences, they 
also have many important attributes in common. In general, they are relatively 
young physicians or scientists with access to microcomputers. They are likely to do 
their own searches. Overwhelmingly they use MEDLINE to satisfy immediate 
information needs rather than to stay current in their fields or to learn about new 
areas. They typically search by subject In general, they are quite satisfied with 
MEDLINE. 

The results of the survey provide objective data on several key questions about 
online searching by individuals. NLM staff views about the system improvements 
most desired by individual users are generally corroborated by the users themselves 
In general, individual users appear to have a very positive view of NLM's online 
service. This is evidenced by the high return rate or the survey and the willingness 
of a large majority of respondents to participate in follow-up studies, as well as bv 
explicit indications of satisfaction. 



vi 



1 



Report on the Survey of 
Individual Users of MEDLINE on the NLM System 



INTRODUCTION 

The National Library of Medicine's basic mission is to improve the dissemination of 
information important to the progress of medicine and to public health. In support of 
this goal, NLM developed MEDLARS, an automated Medical Literature Analysis and 
Retrieval System, in the 1960's and was a pioneer in offering online retrieval services 
over nationwide telecommunications networks in the 1970's. MEDLINE, a file of 
indexed citations to articles in biomedical journals, was the first NLM database to be 
made available online and continues to be the most heavily used MEDLARS file, 
accounting for more than 75% of the over 4 million searches done annually. 

For the first 20 years of its existence, MEDLARS was used principally by medical 
librarians who acted as search intermediaries for health professionals (both researchers 
and practitioners) who needed to locate pertinent information in the health sciences 
literature. To ensure that health professionals received good, cost-effective 
MEDLARS service, NLM required searchers to attend an online services training 
course as a condition of obtaining an access code for system use. 

As more and more individual health professionals began to have access to personal 
computers, NLM took several steps to encourage these individuals to access NLM's 
databases directly as "end-users." These steps included: developing a special one-day 
course in searching MEDLINE for health professionals, training medical librarians 
throughout the U.S. to provide this course in their local settings, streamlining 
procedures for obtaining online access codes, making the system available around the 
clock, and developing a user-friendly microcomputer interface to the NLM system. On 
July 1, 1986, the training requirement was dropped entirely. As a result of these 
actions, the number of individual health professionals accessing MEDLINE directly on 
the NLM system has increased dramatically in the past three years and is continuing to 
grow at the rate of well over 200 users per month. 



PURPOSE OF THE SURVEY 

In order to provide a retrieval service that is both easy to use and effective, NLM 
needed more detailed information about this growing and important end-user 
population, their information needs, and their level of satisfaction with the present 
system, and with MEDLINE, the most frequently used database. 

As a first step, the Library decided to survey the individual users of MEDLINE to 
obtain answers to the following questions: 

1. What are the demographic characteristics of the health professionals (researchers 

and practitioners) who perform their own searches on MEDLINE on the NLM 
computer system? 

2. How do these users enter the system (e.g., directly using the command language or 

via a microcomputer front-end) and why do they use that method? 
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3. For what purposes are they accessing the system (research, patient care, etc.)? 

4. What is their level of satisfaction? 

(a) with the content of the material identified (e.g., relevance of citations, 
completeness of retrieval, data elements available) 

(b) with the search mechanism used 

5. What changes or improvements would make the system more useful to them? 

We hope to use the information gathered to identify system improvements that will 
provide a better level of information service to U.S. health professionals. 



METHODOLOGY 
Questionnaire Development 

In the summer of 1986, a Study Team' was convened to determine what information 
was needed to answer these basic questions and how it might best be obtained. The 
Study Team began by outlining broad categories of interest (e.g., demographics, level of 
training, means of accessing the system, purpose of search, and level of satisfaction), 
and then formed subgroups to develop lists of questions within each of the categories. 
The Study Team developed a draft questionnaire that was circulated to other NLM 
staff for comment With the assistance of a contractor, Market Dynamics, Inc., the 
preliminary questionnaire was restructured and reformatted. A pre-test was then 
conducted on a small group of individual users of MEDLINE on the NLM system. 
Following the pre-test, the questionnaire was revised again and sent to the Office of 
Management and Budget for review as required by the Paperwork Reduction Act. The 
final version of the questionnaire appears as Attachment 1. 

Population Surveyed 

The survey population was defined as individual "end-users' 1 of MEDLINE on the NLM 
system. Since there was no reliable way to define sample strata, the decision was made 
to survey all individuals who had obtained access codes for use of NLM's online system 
in order to obtain as much richness in the data as possible. We were particularly 
interested in qualitative comments that individuals might provide in response to the 
open-ended questions. The only mechanism available to identify the target group was a 
classification system used by NLM's MEDLARS Management Section to process 
applications for online access codes. In cases of doubt, we erred on the side of being 
too inclusive, rather than restrictive, in deciding which classification codes to use.** As 
a result, the mailing included, as best we could determine, all domestic individual end- 
users of MEDLINE on the NLM system as of July 1987. 



•Members of the Study Team were: Patricia Buchan, Betsy Humphreys, Charles Kalina, Sheldon Kotzin, T 
Scott Plutchak, Nancy Selinger, Elliot Siegel, John Starkweather, Carolyn Tilley, Karen Wallingford, and 
Rose Marie Woodsmall. 

** Attachment 5 shows the classification categories and the numbers of individuals in each, and outlines the 
strategy used to define the survey population. 



Instructions were included in both the cover letter and the follow-up letter for those 
who do not search MEDLINE themselves (i.e., who always have an intermediary 
search). They were asked to write across the top of the questionnaire that they do not 
search MEDLINE personally and to return the unanswered questionnaire to NLM. In 
cases in which they had begun filling out the questionnaire, they were instructed to stop 
at the bottom of page 2 after completing only the demographic questions. These 
individuals were not included in the data analyses. 

Data Collection Procedures 

The questionnaires, with a cover letter from the Director, National Library of Medicine 
(Attachment 2), were mailed to 4,311 individuals on October 2, 1987. A reminder 
postcard (Attachment 3) was sent to the entire survey population on October 14. Four 
weeks after the initial mailing, November 3, a follow-up letter (Attachment 4) and a 
duplicate copy of the questionnaire was mailed to those who had not yet responded. 
Returns were accepted until December 10, 1987. 

Questionnaire Return Rates 

The following table shows the questionnaire return rates: 

Mailed Returned % Returned 

4311 
-73' 

4238 2970 70.1% 

-243* -243" 
-11" * 

3995 2716 Usable = 68.0% 

• Undeliverable 

* * Ineligible because not individual users of MEDLINE 

•• 'Unanswered questionnaires returned by eligible individuals 

The return rate is based on the number who actually received the questionnaire. The 
73 undeliverable questionnaires were subtracted from the original number mailed on 
the assumption that they were no longer active individual users of MEDLINE. The 243 
who identified themselves as non-users were also subtracted from the population. The 
11 individuals who erroneously considered themselves as ineligible (because they use 
GRATEFUL MED) were, however, not subtracted from the survey population because 
they are individual users of MEDLINE on the NLM system. The usable return rate was 
thus calculated to be 68.0%. 

Since the universe of individual MEDLINE users was surveyed, sampling error does not 
affect the results. Because the return rate was high, non-response bias is negligible. 
We believe the survey findings to be representative of all individual users of MEDLINE 
as of July 1987. 
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Data Analysis Procedures 

Questionnaires that were returned to NLM by the individuals surveyed were in turn 
sent to the contractor, Market Dynamics, Inc. Market Dynamics was responsible for 
designing a database format to store the questionnaire data and for coding and keying 
the data, with the exception of some of the open-ended questions. Coding of open- 
ended responses that required technical knowledge of the NLM system was done by 
NLM staff. Market Dynamics used the SPSS statistical package for the data analyses. 
Frequency distributions were done for each of the questions and at the outset a number 
of cross-tabs were specified by NLM. As the data were examined, additional cross-tabs 
and t-tests were performed. 

FINDINGS 

The findings from the survey are presented in the following sections: 

o Demographic Characteristics of Survey Respondents 
o General Searching Behavior 
o Characteristics of MEDLINE Searches 
o Opinions about the System 
o Access and Training 
o Comments on the System 
o Willingness to Participate in Follow-up Studies 
o Conclusions 
o Key Points 

Unless, otherwise noted, percentages are based on the number of respondents who 
answered the particular question; individuals who did not answer the question are not 
reflected in the percentages presented. The number of non-responses to each of the 
questions is shown in the tables. 

On several questions, respondents were instructed to give more than one response, if 
appropriate. In these cases, percentages will sum to more than 100%. 

Demographic Characteristics of Survey Respondents 

Professions (Table 1) 

Question 1 asked respondents to indicate their profession(s) by circling all that applied. 
2,710 individuals (99.8% of the entire survey group) responded to the question. Nearly 
two-thirds identified themselves as physicians (65.5%) and more than a quarter (27.7%) 
as scientists. 482 individuals did indicate more than one profession; 286 (10.5%) said 
that they were both physicians and scientists. Thus, 82.7% of the respondents who 
indicated their profession are physicians or scientists, or both. 

Nurses made up 3.7% of the respondent group. Another 6.6% said they were "other 
health professional." 7.6% listed their profession as "other." Included in this group 
were diverse professionals such as lawyers, statisticians, engineers, etc., but the numbers 
in the various "other" professions are too small to break out into separate categories 
4.4% said they were students. Although the survey was intended for the population of 
users who search MEDLINE as individuals ("end-users"), 3.2% of the respondents 
indicated that they were librarian/information specialists. Of these, 28% indicated 



more than one profession: nine were also "other health professionals," four were 
physicians, four nurses, four "other," and three students. 

Because the respondents were able to indicate more one than profession, the 
percentages total more than 100%. 

Specialties (Table 2) 

Question 4 asked the respondents who are health professionals to indicate their 
specialties. 2,197 (80.9% of the entire respondent group) answered this question. 
Those who responded are involved in a broad range of medical specialties. More than 
40 specialties were listed, with some respondents indicating more than one specialty. 
The specialties listed most often are: internal medicine (282), medical oncology (147), 
pathology (125), and pediatrics (109). The full distribution of specialities is given in 
Table 2. 

Education (Table 3) 

Question 2 asked the respondents to indicate the highest degree held. The majority of 
the respondents hold either M.D. or Ph.D. degrees. 1,760 said that they hold M.D. 
degrees, and 570 hold Ph.D. degrees. Although the question asked for the highest 
degree (implying a single response), some respondents did list more than one degree. 

Year Highest Degree Was Received (Table 4, 5) 

Question 3 asked the respondents to indicate the year in which they received their 
highest degree. Of the group, 982% (2,667 individuals) answered this question. 
Answers spanned a range of 60 years, with one person receiving the highest degree in 
1929, and one expecting a degree in 1989. The majority received their highest degree 
within the last 18 years: 40.1% during the 1970's and 27.1% in the 1980's. Thus, the 
majority of individual users of MEDLINE appear to be relatively young. Just under 
20% (19.6%) received their highest degree in the 1960's and 10.1% m the 1950's. Only 
3.1% received their highest degree prior to 1950. 

Among the various professional groups, over half of the nurses (53.5%) and almost 
three-fourths of the students (72.8%) received their highest degrees in the 1980's. 

Primary Work Place (Table 6, 7) 

Question 5 asked respondents to indicate their primary work place by circling only one 
of the choices provided or by specifying an "other." Nearly the entire group (99.4%, 
2,701 individuals) answered, almost half (46.8%) indicating they work in an academic 
setting (college, university, or medical school). The remaining respondents are split 
evenly among private/solo practice (12.6%), group practice (12.3%), and 
hospital/clinic (14.6%). Only 5.4% work in a government agency, and 3.3% in a private 
company or business. Nearly half of the physicians (49.7%) work in a clinical practice 
setting (group practice 17.7%, private/solo practice 16.6%, and hospital/clinic 15.4%). 
40.4% work in academic settings. 

Over three-fourths of the scientists work in academic settings. The largest percentage 
(45.8%) of the "other health professional" group also work in academic settings. Nurse 
respondents tend to work in either an academic setting (41.4%) or hospital/clinic 
setting (323%). 
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General Searching Behavior 



Frequency of Searching 

Question 8 asked the respondents to indicate the number of MEDLINE ^^jj^JlS 
do by themselves in the average month. 98.9% (2,686) of the respondents answercu i 
question. More than two-thirds (68.6%) indicated that they do all scarcnca , j 
themselves. Half (52.7%) report that they perform two or fewer searches in l tne 
month. Included in this group are the 12% who report that they don t P*™ 1 ™" £g 
searches in the average month. These individuals were included in tne 
respondent group because, while they may not perform any searches in tn b 
month, they do occasionally search MEDLINE by themselves. More JBan a uuxu 
(34.1%) perform 4 or more searches in the average month. Of the respondent i who 
reported that they perform at least one search m the^ average month, tne average 
number of searches performed is 43 searches per month.* 

Question 9 asked respondents to indicate the number of searches : thev have someone 
else do for them in the average month. Again, 98.9% (2,686 *^^\^*lf?™ 
question. The majority (68.6%) entered a zero, indicating that they do all ot their own 
searches. Of the one-third who at times have others perform searches tor them, u.i /o 
have one search per month, 10.2% have 2-3 searches, and only 6.0% have 4 or more 
searches done by someone else in the average month. For those who have others 
perform searches for them, the average number is 25 per month. 

Searches Performed by Others (Table 10-A/B, 11-A/B, 12, 13, 14) 

Question 10 was a multi-part question that asked respondents who occasionally or 
always have someone else search for them to indicate: A) who generally does the 
searches; B) under what circumstances is it preferable to have someone else search; C) 
what is the level of satisfaction with searches done by self and by others; and D) if 
generally not satisfied with searches done by others, what are the reasons for 
dissatisfaction. 

In response to the first part of the question, 1,103 individuals (40.6% of the entire 
respondent group) indicated that they at least occasionally have someone else perform 
searches for them. The majority (81.9%) of the searches performed by others are done 
by librarian/information specialists. Librarians tend be selected most frequently by 
those working in a government agency (89.7%), hospital/clinic (89.5%), group practice 
(89.4%), and private/solo practice (85.5%). Respondents in an academic setting have 
searches performed by student/research assistants (10.5%) and colleagues (6.7%), as 
well as librarians (74.1%). 

In response to part B of the question, 1,151 individuals (42.4% of the entire respondent 
group) provided the reasons why they, at least occasionally, have someone else search 
for them. "When I don't have time to do it myself" was the most frequently cited reason 
for having someone else perform MEDLINE searches. 59.3% of the responses 

•Because we were concerned that the few librarian/information specialists included in the respondent 
group might have inflated the average number of searches per month, we did some further analyses that 
excluded the librarian/information specialists. These analyses indicated that although the 
librarian/information specialists do perform far more searches (13.2) in an average month, they have a 
negligible effect on overall results. In addition, as noted earlier, over a quarter (28.0%) of the individuals 
who listed their profession as librarian/information specialist also indicated another profession. 



indicated this reason. Slightly more than a third (38.7%) had someone else search 
when they needed different expertise or system knowledge. 27.5% had someone else 
search after they had tried a search and not been satisfied with the results. 
Respondents to this question were instructed to circle all that apply, and in this case 
percentages again exceed 100%. 

1,121 individuals (41.3% of the entire respondent group) indicated their level of 
satisfaction with searches done for them by someone else on a scale of 1 (very satisfied) 
to 5 (not at all satisfied). 70.6% (819 individuals) expressed satisfaction with the results 
by indicating 1 or 2, 8.9% (100 individuals) are dissatisfied (responses of 4 or 5), and 
20.5% (230 individuals) are neutral (response of 3). 

Compared to the percentage of satisfied individuals from the respondent group as a 
whole, higher percentages of individuals in government (80.6%), private/solo practice 
(74.2%) and group practice (74.1%") reported being satisfied with searches clone by 
others. Slightly lower percentages of individuals satisfied with searches done by others 
were found in academic (68.5%) and hospital/clinic settings (65.5%). Private business 
not only had the highest percentage of satisfied individuals (80.7%), but also the highest 
percentage of dissatisfied respondents (12.9%). 

150 individuals indicated reasons for dissatisfaction with searches done by others. 
Respondents to this part of the question were instructed to circle all that applied, and 
as a result, percentages exceed 100%. Almost two-thirds of this group (62.0%) cited 
dissatisfaction with the results of the search. 393% stated that they have to wait too 
long for search results. "Other" was the next most frequent reason given for 
dissatisfaction (30.7%). The fourth most frequently listed reason was cost (24.0%). 

Searches Performed by Respondents 

Whv individuals search (Table 15) 

Question 11 asked respondents to circle all factors that influenced their decision to 
search MEDLINE themselves rather than having someone else search for them, and 
then to check the single most influential factor in their decision. 2,171 individuals 
(79.9% of the entire respondent group) answered this question; because multiple 
answers were allowed, percentages exceed 100%. Greater familiarity with the subject 
matter was selected by 83.1% of the respondents, and the ability to get results faster by 
82.4%. Almost two-thirds of the respondents (65.5%) indicated enjoyment of searching 
as a factor. Nearly a third (31.6%) said that one of their reasons for searching by 
themselves is that it is more cost effective than using an intermediary. Only 14.7% cited 
lack of an intermediary as one of the reasons for doing their own searches. 

2,074 individuals (76.4% of the entire respondent group) indicated a most influential 
factor. As the single most influential factor, respondents indicated greater familiarity 
with the subject matter than search intermediaries possess (47.9%) or the ability to get 
search results faster (32.2%). 7.5% indicated enjoyment of searching as the most 
influential factor in the decision to search for themselves. 

Level of experience (Table 16, 17) 

Question 12 asked respondents to indicate how experienced a user of online databases 
they considered themselves to be. Choices ranged from 1 (very experienced) to 4 (not at 
all experienced). 2,561 individuals (94.3% of the entire respondent group) answered 
this question. Less than 10% of the respondents felt themselves to be very experienced 
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users of online databases. The majority (84.5%) felt they were somewhat experienced 
or not very experienced. Less than 8% (7.2%) indicated that they were not at all 
experienced. 

Pattern of use of MEDLINE (Table 18, 19, 20, 21, 22) 

Question 13 asked respondents to indicate how long they have been searching 
MEDLINE using their own code. 2^09 (92.4% of the entire respondent group) 
individuals responded. Nearly two-thirds (63.1%) have had their codes for a year or 
less. Just under another third (27.0%) have had their codes between one and two years. 
Only 10.0% have had access codes for more than two years. 

Individuals who always use command language to search have, on average, had their 
codes twice as long (1.54 years vs. 0.8 years) as those who always search using the 
GRATEFUL MED form screens. 

Question 7 asked respondents to indicate how many people (including themselves) 
share their access code. Respondents were instructed to enter aT if they were the 
only person who uses the code. 2,661 individuals (98.0% of the entire respondent 
group) answered this question. Overall, the respondents do tend to be true individual 
users of the system with the majority (82.0%) indicating that they don't share a code 
with anyone else. 8.6% share a code with one other person and another 8.7% share a 
code with two or more people. 

Question 14 asked respondents whether their use of MEDLINE has increased or 
decreased since they received their own access codes. 2,545 individuals (93.7% of the 
entire respondent group) answered this question. More than half (57.7%) said that 
their use has stayed the same. 30.0% indicated increased use and 123% said that then- 
use has decreased. 



Question 15 was an open-ended response question that asked individuals whose usage 
had either increased or decreased to indicate the reasons for the change. The two most 
cited reasons given by the 678 individuals who reported increased use: increased 
familiarity with the system and greater need. Decreased need and lack of time were the 
two reasons cited most frequently by the 288 individuals who indicated their use of 
MEDLINE had decreased. 



Cost considerations (Table 23, 24) 

Question 16 asked respondents to indicate how often cost considerations keep them 
from doing a MEDLINE search. 2^54 individuals (94.0% of the entire respondent 
group) responded. Cost seems not to be of overwhelming importance in individuals' use 
of MEDLINE on the NLM system. The majority (67.6%) indicated that cost 
considerations rarelv (26.6%) or never (41.0%) keep them from performing a search 
Almost one-fourth (23.1%) said that cost considerations occasionally keep them from 
searching. Only 9.3% reported that cost considerations frequently keep them from 
searching. 

Those most constrained by cost considerations reported having no formal work Dlace 
with 46.1% of that group citing cost as a frequent or occasional constraint Sliehtlv nver 
a third of those individuals in an academic (36.0%) or hospital/clinic (34 6%} settino 
said that cost was frequently or occasionally a consideration in searching. cuing 
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Cost seems to be less of a consideration for those working in group practice, private 
business, and private/solo practice, than it is for those in academic and hospital/clinic 
settings. Under a third of those working in private business (20.6%), group practice 
(22.6%) and private/solo practice (27.4%) reported that cost considerations 
occasionally or frequently are constraints on searching. 

Characteristics of MEDLINE Searches 

General Search Purpose (Table 25, 26) 

Question 18 was a multi-part question that asked respondents about their most typical 
reasons for searching MEDLINE. 

2^56 individuals (94.1%) answered the first part of the question. Four-fifths (81.7%) of 
the respondents indicated they typically search MEDLINE to satisfy an immediate 
information need. The highest percentages in this category are found in group practice 
(883%), followed by those who indicated "other" work place (85.2%), pnvate/solo 
practice (83.6%), and government agency (83.2%). 

Ten percent of all respondents typically search for information to stay current in their 
field. Those in private business had the highest percentage (17.2%) indicating this as 
their typical reason for searching MEDLINE. People who indicated no formal work 
place (15.4%) and individuals in academic settings (12.6%) ranked second and third. 
Only 7-8% of those in private/solo or group practice or in a hospital/clinic are typically 
interested in searching for information in order to stay current in their fields. 

Only 5% of all respondents indicated they typically search to learn about new areas. 
The highest percentage (6.6%) are individuals in a government agency; only 1.9% of 
those in group practice and "other" work places "typically search to learn about new 
areas. 

Type of Retrieval Desired (Table 27, 28, 29) 

The second part of the question asked respondents to indicate the number of citations 
they typically wish to retrieve (all relevant or just a few relevant citations). 2,550 
individuals (93.9%) responded. The majority (58.9%) wish to retrieve all relevant 
citations from a particular time period rather than just a few relevant citations. Among 
the various work place groups, the percentages of individuals who typically want to 
retrieve all relevant citations range from a high of 65.2% of those in academic settings 
to a low of 47.6% of those in group practice. Analyzed by primary area in which search 
information is used, the percentages of individuals interested in all relevant citations 
ranges from a high of 67.1% of those who most commonly use search information for 
research/testing purposes to a low of 49.8% of those who most commonly use search 
information for patient care. 

Numbers of Citations (Table 30, 31, 32, 55) 

The third part of question 18 asked respondents whether they typically retrieve too few 
citations, about the right number of citations, or too many citations. 2,510 individuals 
(92.4% of the entire respondent group) answered this part of the question. 61.1% of 
the respondents indicated that they typically retrieve about the right number of 
citations. When analyzed by work place, the groups with the highest percentages 
indicating retrieval of the right number of citations are: "other" work place (73.1%) and 
government agency (69.7%). 
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Overall, 15% of the respondents feel that they typically retrieve too few citations. The 
highest percentages choosing this response are found in private business (23.5%), 
pnvate/solo practice (21.7%), and group practice (20.6%), even though almost half of 
the respondents in these same work places expressed interest in retrieving only a few 
relevant citations. 



Just under a quarter (23.6%) of all respondents feel that they retrieve too many 
citations. A slightly higher percentage of those individuals in private business (27.1%) 
indicated this. 

Over 70% of all respondents always use either the command language or GRATEFUL 
MED form screens (43.2% and 26.9%, respectively) to search MEDLINE. Of the 
individuals who always search using command language, 64.8% indicated that they 
typically retrieve the right number of citations, compared to 56.9% of the individuals 
who always use the GRATEFUL MED form screens. 

A higher percentage of those who always search using GRATEFUL MED form screens 
retrieve too few citations compared to individuals who always search using command 
language (23.6% vs. 10.2%). 

In contrast, 25.0% of the respondents who always search using command language 
indicated that they retrieve too many citations compared to 19.6% of the searchers who 
always use GRATEFUL MED form screens. 

Relevance of Citations Retrieved (Table 33, 34) 

The final part of question 18 asked respondents to indicate what percentage of the 
citations they typically retrieve are relevant to their inquiry. 2,399 individuals (88.3% of 
the entire respondentgroup) answered this portion of the question. More than half of 
all respondents (58.6%) feel that of the citations they typically retrieve, fewer than half 
are relevant. Analyzed by work place, an even higher percentage of those in group and 
private/solo practice feel this way (63.4% and 61.1%). 

Of the group of respondents as a whole, 41.4% feel that half or more of the citations 
typically retrieved are relevant. Individuals working in "other" places (52%) private 
business (48.8%), government agency (47.3%), and those with no formal work place 
(45.5%) were slightly more inclined to report retrieving greater percentages of relevant 
citations. 

Types of Searches Conducted (Table 35) 

Question 19 asked respondents whether they most often search for an author a iournal 
title, or a subject. Instructions specified that only one choice should be circled 2 552 
individuals (94.0% of the enure respondent group) answered this question 9*6 0% of 
the respondents said that they most often search for a subject. Only 3 2% indicated th^t 
they most often search for an author and less than 1% said that they most often se-irrh 
for a journal title. J *««uac*rcn 

Use of Information Retrieved (Table 36-A/B/C, 37, 38) 

Question 20 asked respondents to rank the three primary areas in whirh th~, 
MEDLINE search information (1 = most common, 2 
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Regardless of the rank assigned, the areas mentioned most frequently were the 
following: research/testing (71%), patient care (69.1%), and education (66%). 

2^92 individuals (95.4% of the entire respondent group) assigned a rank of 1 to 
indicate the area in which search information is most commonly used. 45.8% ranked 
research/testing as the most common use, followed by 38.2% who indicated patient 
care as tne most common use. Educational use was selected by 12.4%. 

Of all individuals who listed research/testing as one of their uses of MEDLINE 
information, 61.5% indicated it as the most common use. Similarly, of all individuals 
who ranked patient care as one of their uses, 52.8% ranked patient care as the most 
common. In the area of education the results are very different: of the individuals who 
listed education as one of their uses of MEDLINE information, only 17.9% indicated it 
is their most common use. 

Of the individuals in private/solo practice, over two-thirds (68.8%) indicated that their 
most common use of the information is for patient care. Similarly, 75.8% of those in 
group practice most commonly use MEDLINE information for patient care. One-half 
of those in a hospital/clinic setting indicated patient care as their most common use. In 
contrast, of the individuals in academic settings, two-thirds (66.9%) indicated their most 
common use of MEDLINE information is for research/testing purposes. Over half of 
the individuals in private companies/businesses and government agencies also indicated 
that their most common use is for research/testing (52.8% and 56.6%, respectively). 
Only about one-tenth of the individuals in private/solo practice (10.6%), group practice 
(8.5%), and academic settings (10.3%) listed educational purposes as theu* most 
common use of MEDLINE information. Higher percentages of individuals in 
hospital/clinic setting (19.2%) and private company /business (15.7%) indicated 
educational purposes as their most common use. 

Of all respondents who listed patient care as their primary use of MEDLINE 
information, 40.8% always use command language to search while 31.6% always use the 
GRATEFUL MED form screens. Twice as many individuals who always use command 
language search for research/testing purposes as compared to GRATEFUL MED form 
screen users (45.6% vs. 23.1%). And far more of the respondents whose primary use of 
MEDLINE information is for management/administration purposes indicated that they 
always use command language compared to individuals who always use the form 
screens (60.7% vs. 10.7%). Similarly, of all respondents who use MEDLINE 
information for regulation purposes, 50% always use tne command language vs. 12.5% 
who always use the GRATEFUL MED form screens. 

Use ofMeSH (Table 39, 40-A/B, 41) 

Question 21 asked individuals to indicate how often they use MeSH terms when 
searching for specific subjects. Choices provided were: 1. Always, 2. Usually, 3. 
Occasionally, 4. Rarely, 5. Never. 2,544 individuals (93.7% of the entire respondent 
group) provided a response to this question. Over two-thirds of the respondents 
(67.7%) always use (24.1%) or usually use (43.6%) MeSH terms for searching for 
subjects in MEDLINE. 8.8% rarely and 5.1% never use MeSH. There is a similar 
distribution of responses to question 22 which asked respondents to indicate how useful 
they find the MeSH terms to be. The scale provided ranged from 1 (very useful) to 5 
(not at all useful). 2,475 individuals (91.1% of the entire respondent group) provided 
an answer to this question. Two-thirds find MeSH terms very useful (26.7%) or useful 
(39.0%), while 7.8% say that they're not useful and 3.2% say they're not at all useful. 
The remaining 23.2% are neutral regarding the usefulness of the MeSH terms. 
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When considered by work place, 73.2% of those in group practice and 71.7% of those in 
a hospital/clinic tend to use MeSH. Correspondingly, higher percentages of individuals 
in group practice (70.0%) and hospital/clinic settings (73.3%) feel that MeSH terms are 
useful. 

Among the few who indicated that they rarely or never use MeSH, the highest 
percentages are found in government agencies (20.6%) and in private business (19.5%). 
Higher percentages of the individuals in these same work places (government agencies 
19.4% and private business 15.3%), along with individuals with no formal work place 
(16%) said that MeSH terms are not very or not at all useful. 

A t-test was done to compare the individuals who always search using the command 
language (N = 1,060) with the individuals who always search using the GRATEFUL 
MED form screens (N=653) with regard to their use of MeSH terms. The mean for the 
group of individuals who always search using the command language is 2.08, indicating 
that the individuals who always use the command language usually use MeSH terms to 
search, while the mean for the group who always search using the GRATEFUL MED 
form screens is 2.46 (p = .000), indicating that among these individuals MeSH terms are 
used less frequently in searching MEDLINE. These two groups were also compared on 
how useful they find the MeSH terms to be. In the group who always search using the 
command language and who answered question 22 (N = 1,051), the mean is 2.14 
indicating that they generally find MeSH terms to be useful, while for the group of 
individuals who always use GRATEFUL MED form screens (N=620), the mean is 226 
(p < .05), indicating that these individuals, on average, think MeSH terms are useful, 
but less so than the group who search using the command language. 

Question 23 was an open-ended question that asked those individuals who think MeSH 
terms are generally not useful or who never use MeSH to indicate why. Responses 
were provided by 454 individuals (16.7% of the entire respondent group). The two 
reasons given most frequently for not using MeSH are: unfamiliarity with MeSH 
(N=116) and that MeSH terminology is cumbersome to use (N=107). The next two 
most frequently given reason for not using MeSH is that "there are no terms in my area" 
(N = 57), followed by "terms are too general" (N = 49). 

Length of Searches (Table 42) 

Question 24 asked how long it typically takes to search MEDLINE for citations on a 
particular subject 2,533 individuals (93.3% of the entire respondent group) answered 
this question. For almost two-thirds of the respondents (64.1%), a typical search takes 
less than 10 minutes. Another quarter (25.2%) typically take between 10-15 minutes to 
conduct a search. 10.7% typically take more than 15 minutes. 

Question 25 then asked whether the time was too long, reasonable or Quicker than 
expected. £526 individuals (93.0% of the entire respondent ^oSp TanWed thk 
question. The majority of the respondents (89.1%) feel that the Kf time to 
conduct a search is reasonable, including 5.7% who find it to be quicker than *™ZJa 
Only 10.9% think it takes too long. quicxer man expected. 
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Opinions about the System 

Level of Satisfaction with Searches (Table 43, 44) 

Question 8 asked individuals to indicate on a scale of 1 (very satisfied) to 5 (not at all 
satisfied) how satisfied they are with searches they do themselves. 1,318 individuals 
(48.5% of the entire respondent group) answered this question. Almost three-quarters 
(72.5%) of these respondents are satisfied with searches they do for themselves. 
Compared to the group overall, higher percentages of those with no formal work place 
(90.0%\ those in a government agency (79.3%), and those in an academic setting 
(763%) express satisfaction with searches they do themselves. 

Just under ten percent (9.0%) say that they are not satisfied with searches they do for 
themselves. Those in private/solo practice (14.7%) had the highest rate of 
dissatisfaction with their own searches. 

Overall Satisfaction with MEDLINE (Table 45, 46, 47) 

Using a scale of 1 (very satisfied) to 5 (not at all satisfied), question 17 asked the 
respondents to rate their overall satisfaction with MEDLINE on the NLM system. 
2,5j9 individuals (93.5% of the entire respondent group) answered this question. 
Overall, 81.4% of the respondents indicated satisfaction with MEDLINE. Only 4.1% 
indicated that they are not satisfied. The remainder are neutral, neither satisfied nor 
dissatisfied. 

Analyzed by work place, the highest percentages of satisfied individuals are those with 
no formal work place (92.3%) and those working in a government agency (90.5%). The 
groups expressing the highest percentages of dissatisfaction are those in private/solo 
practice (6.7%) and group practice (5.8%). 

Individuals who always use GRATEFUL MED form screens and individuals who 
always use the command language to search appear to be equally satisfied with 
MEDLINE (823% and 82.5% respectively). Similarly, 3.6% of the individuals who 
always use command language report being dissatisfied with MEDLINE compared to 
3.5% of those who always use the GRATEFUL MED form screens. 

Desirable Additions to MEDLINE (Table 48, 49, 50) 

Question 26 asked respondents whether there are types of information that they would 
find valuable that they cannot routinely find in a MEDLINE citation. 60.8% of the 
respondents answered no, indicating that MEDLINE citations are acceptable in their 
current form. Individuals who answered yes to the question were then presented with a 
list of information types from which to choose. Respondents were instructed to circle 
all that would be valuable, and to check the single most valuable type of information. 

Among the respondents who indicated that there are other types of information they 
would find valuable (N = 1,019, 37.4% of the entire respondent group), full text of 
articles was the overwhelming choice; 79.1% listed it as one of their choices and 65.2% 
as the single most valuable information to add to MEDLINE. Author address was the 
next most frequently mentioned (37.0%), with 11.2% of the respondents indicating it as 
the most valuable piece of information. 31.4% said that research design would be 
valuable, but only 7.9% chose it as the single most valuable. Similarly, 23.6% said 
they'd find dosage information valuable, but only 4.5% thought it would be the single 
most valuable type of information to add to MEDLINE citations. 
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When viewed by primary purpose of search (patient care, education, research/testing), 
just under two thirds of those primarily interested in patient care (64.0%, N=304) ana 
education (62.4%, N=83) selected full text as most valuable as opposed to just unaer 
half of those interested in research/testing (48.0%, N=271). Dosage information was 
the next piece of information most frequently identified *s most valuable by those witn 
a primary interest in patient care (6.3%, N=30) and education (5.3%, N=30, tied wan 
journal section); those interested in research/testing selected author address (lo.i/o, 
N=91). The piece of information which ranked third as most valuable was research 
design, chosen by 9.4% (N=53) of those interested in research/testing and 5.3% 
(N=25) of those in patient care. 

Desirable Improvements to MEDLINE (Table 51, 52, 53) 

Question 27 asked respondents to indicate the features or capabilities they would like 
to see added to MEDLINE by selecting from a list provided and to check the one 
feature they would most like to see. 2,139 individuals (78.8% of the entire respondent 
group) provided an answer to this question. 71.8% of the respondents indicated 
unproved MEDLINE backfile searching and 69.2% want improved capabilities for 
printing citations. 

These same two capabilities were chosen most frequently as the single most desired 
capabilities individuals would like to see added to the NLM system: 35.9% listed 
improved backfile searching as the most desired capability, and 27.9% chose improved 
capability for printing citations as the most desired. 

Improved backfile searching and improved print capability were the overwhelming first 
and second choices irrespective of the respondents' typical purpose of search. Among 
the individuals who typically search for educational purposes, 32.2% indicated more 
"didactic" literature as one among the features they would like to have added to 
MEDLINE and 9.8% considered it the most desirable additional feature. In the group 
who typically search for patient care purposes, 21.5% listed more "didactic" literature 
among their choices for features to add to MEDLINE. 

Individuals who always search using command language listed improved backfile 
searching (70.1%) and improved print quality (71.1) most frequently among all desired 
capabilities. Individuals who always search using GRATEFUL MED form screens also 
chose the same two features: backfile searching (63.8%) and printing (55.6%), although 
in smaller percentages. 

Access and Training 

Equipment (Table 54) 

Question 6 asked whether individuals have microcomputers available in their work 
5i a ££ mdi Yi duals .( 99 - 7% of toe entire respondent group) answered this question, 
96.0% of them affirmatively. Question 28 then asked respondents to indicate whether 
they pnmanly use a microcomputer or a terminal to access MEDLINE. Of the 2 556 
individuals (94.1% of the entire respondent group) who provided an answer 96 0% 
said that they use a microcomputer, only 4.0% indicating that they use a terminal 
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Command Language Searching vs. Use of GRATEFUL MED 
(Table 55, 56, 57, 58) 



Question 29 asked respondents to indicate the percent of MEDLINE searches they 
performed using various access methods. Instructions stated that the percent should 
add to 100 and that if a method was not used, respondents should enter a "0" in the 
blank provided. 2,474 individuals (91.1% of the entire respondent group) answered this 
question. 1,870 individuals (75.5% of those who answered the question) indicated that 
they always use a single method to access MEDLINE, almost half (43.2%) always using 
the command language, while over one-fourth (26.9%) always use the GRATEFUL 
MED form screens. Smaller percentages of individuals always use another front-end 
package (4.2%) or GRATEFUL MED direct option (1.8%). Only 604 individuals 
indicated that they use more than one method to access MEDLINE. Of these, 504 use 
GRATEFUL MED form screens some of the time, 359 use GRATEFUL MED direct 
option some of the time, 268 use command language some of the time, and 124 use 
another front-end package some of the time. The remainder of this Section presents 
data only on those individuals who either always search using the command language or 
who always search using the GRATEFUL MED form screens. 

Higher percentages of students (503%), other health professionals (49.4%), nurses 
(47.6%), and scientists (46.6%) always use the command language to search than of 
physicians (39.7%). More of those reporting no formal work place (57.7%) or "other" 
work place (47.1%) always search using command language as compared to individuals 



A larger percentage of the physician group (30.1%) always uses the GRATEFUL MED 
form screens in comparison to the other professional groups ("other health 
professional" 24.8%, scientist 21%, nurse 18.5% and student 17.1%). One-third of the 
respondents in private/solo and group practice use GRATEFUL MED form screens all 
of the time. 

A greater percentage of those who are younger (receiving degrees in the 1970's and 
1980's) search MEDLINE using the command language, while a higher percentage of 
those receiving their degrees earlier search using the GRATEFUL MED form screens. 
Half of those receiving degrees in the 1980's indicated that they always search 
MEDLINE using the command language, whereas only 25.0% of those who received 
their highest degree before 1950 do so. The percent who search using command 
language increases as age decreases: only 41.2% of those receiving degrees before 1950 
sometimes search using the command language, as opposed to 61.3% of those who 
received degrees in the 1980's. The reverse is true for use of the GRATEFUL MED 
form screens. 60.9% of those receiving degrees before 1950 use them at least some of 
the time, compared to 30.7% of those receiving degrees in the 1980's. 

Difficulties Accessing MEDLINE (Table 59) 

Question 31 asked respondents about the types of problems, if any, they have in 
accessing the NLM computer. 2,392 individuals (88.1% of the entire respondent group) 
providea an answer to this question. Respondents were instructed to circle all choices 
that applied. More than half of the respondents (53.8%) reported no problems in 
accessing the NLM computer. The most frequent problem encountered is busy 
telecommunications lines (19.5%). 
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Learning to Search MEDLINE (Table 60, 61-A/B, 62-A/B, 63-A/B, 64-A/B) 

Question 32 asked individuals to indicate all the methods they used to learn jo search 
MEDLINE on the NLM system, and to check the one method that was me > moM 
helpful. 2,203 individuals (81.1% of the enure respondent group) answer «a 
question, with 2,059 individuals (75.8% of the entire respondent group) listing a merauu 
that they found most helpful. The most frequently mentioned methods were. £ ljvi- 
sponsored training course (49.7%), using GRATEFUL MED (453%), and sclf-taugnt 
(45.0%). These same three methods were most frequently listed as most neiprui. 
NLM-sponsored training (42.7%), use of GRATEFUL MED (313%), and self-taught 
(11.8%). 

Among the individuals in both private/solo practice and group pra^ice, GRATCFUL 
MED ranked first among the methods used to learn to search MEDLINfc w*-* /0 . ana 
51.2% respectively). Self-taught (443% and 44.5%) and attended NLM training 
(412% and 44.1%) were the next most frequently used methods by individuals m 
private/solo and group practice. Individuals m academic and hospital/clmic settmgs 
most frequently indicated that they attended NLM training (53.6% and 51.4%), 
followed by self-taught (43.8% and 45.9%), and used GRATEFUL MED (42.1% and 
443%). 

Among the individuals who always search using the command language, NLM training 
was most frequently (72.8%) listed as a method used to learn to search MEDLINE, 
followed by self-taught (44.4%), and taught by a co-worker (14.0%). In this group of 
individuals, NLM training was most frequently selected as the most helpful method by 
68.5% of the respondents. 

In the group of individuals who always search using the GRATEFUL MED form 
screens, GRATEFUL MED was by far the most frequently cited (93.1%) among all 
methods used to learn to search, followed by self-taught (38.7%) and NLM-sponsored 
tr ainin g (17.7%). In this group, 82.2% selected GRATEFUL MED as the most helpful 
method in learning to search. 

A t-test was done to compare the group of individuals who always search using the 
command language with those who always search using the GRATEFUL MED form 
screens in regard to whether they had attended either the 3-5 day training class offered 
by NLM or the 6-hour Basics of Searching MEDLINE course. 19.9% of the individuals 
who always search using the command language had attended the 3-5 day class 
compared to 2.7% of the individuals who search using GRATEFUL MED form 
screens. And twice as many individuals who always search using the command language 
attended a 6-hour Basics class compared to those individuals who always use the 
GRATEFUL MED form screens (29.7% vs. 14.7%). 

Question 33 asked those respondents who had attended the 3-5 day training course 
and/or the 6-hour Basics of Searching MEDLINE course to indicate their level of 
satisfaction with the training. Again, the scale ranged between 1 (very satisfied) and 5 
(not at all satisfied). Of those who said that they took the 3-5 day training course (328 
individuals, 12.1% of the entire respondent group), 83.8% indicated that they were 
satisfied with the training. Less than 3% were dissatisfied. The rest were neutral. 
Satisfaction with training does relate to overall satisfaction with MEDLINE; 83.7% 
indicated that they are satisfied with both the training and with MEDLINE. Similarly, 
2.8% indicated dissatisfaction with both the training and with MEDLINE. 13.6% were 
neutral on both. 
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Of those who took the 6-hour Basics of Searching MEDLINE course (1040 individuals, 
383% of the entire respondent group), 83.5% were satisfied with the training they 
received. Less than 4% of this group indicated dissatisfaction. The rest were neutral. 
Again, satisfaction with training relates to overall satisfaction with MEDLINE; 83.7% 
indicated satisfaction with both the 6-hour course and with MEDLINE; 3.5% indicated 
dissatisfaction with both the training and with MEDLINE. 12.8% were neutral on both. 

Comments on the System 

Most Satisfactory Aspects (Table 65, 66) 

Respondents were given space to comment on their perceptions of the most satisfactory 
aspects of MEDLINE. Comments were provided by 747 individuals (27.5% of the 
entire respondent group). Percentages are based on the number of individuals 
responding, rather man the number of aspects they cited. The most frequently cited 
aspect is the content of MEDLINE (32.7%). Hours and availability (25.4%) and speed 
and efficiency (24.6%) were the next most frequently mentioned. GRATEFUL MED 
(19.9%) was fourth on the list, followed closely by cost (18.7%). 

Among the individuals who always search using the command language and who 
provided comments on the aspects they find most satisfactory (N=305, 11.2% of the 
entire respondent group), the following were most frequently cited: content of the 
database (39.8%), speed and efficiency (293%), hours and availability (27.6%), and 
cost (24.7%). Of those who always search using the GRATEFUL MED form screens 
and who provided comments on the most satisfactory aspects (N = 194, 7.1% of the 
entire respondent group), the aspects most frequently cited are: GRATEFUL MED 
(45.9%), hours and availability (253%), content of the database (23.2%), and speed 
and efficiency (21.6%). Cost was mentioned by only 12.9% in this group. Again, 
because a single respondent may have indicated more than one aspect, percentages 
total to more than 100%. 

Least Satisfactory Aspects (Table 67, 68) 

Respondents were also given space to comment on aspects of MEDLINE that they 
considered to be least satisfactory. Comments on least satisfactory aspects were 
provided by 874 individuals (322% of the entire respondent group). Topping the list 
were GRATEFUL MED limitations (17.2%), backfile searching (14.0%), and MeSH 
vocabulary/indexing (13.7%). 

Among those who always search using the command language, and who provided 
comments on the aspects they find least satisfactory (N=387, 14.2% of the entire 
respondent group), the aspects cited most frequently are: MeSH vocabulary/indexing 
(1X3%), command language searching (16.0%), printing (16.0%), and backfile 
searching (13.2%). "GRATEFUL MED not available for other types of 
microcomputers" was mentioned as a least satisfactory aspect by 12.9% of the command 
language users who provided comments. 

In the list of least satisfactory aspects cited by individuals who always use GRATEFUL 
MED form screens (N = 196, 7.2% of the entire respondent group) are: GRATEFUL 
MED limitations (37.2%), backfile searching (15.3%), and MeSH vocabulary/indexing 
(10.7%). 
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Willingness to Participate in Follow-Up Studies (Table 69) 

86.1% of the 2^42 individuals (86.2% of the entire respondent group) who smswered 
this question indicated that they would be willing to participate in follow-up stuaies, 
while only 13.9% declined. 374 individuals (13.8%) did not answer the question. 

CONCLUSIONS 

In the two years before the survey was conducted, NLM took two major : steps to 
encourage individuals to search MEDLINE directly. The first was the establ^hment in 
1985 of a program to train health professionals in the basics of searching MEDLINE on 
the NLM system. To support this program the Library developed a 6-hour course and 
trained medical librarians throughout the United States to teach it The widespread 
availability of this short course led to the first sharp increase in the number of 
individual users of MEDLINE. About 25% of all survey respondents (27% of those 
who supplied the information) indicated that they obtained their codes during the first 
year the course was given, i.e., 13 to 24 months prior to July 1987. 

The second step was the introduction in March 1986 of GRATEFUL MED, a PC 
software package that allows users without special training to search MEDLINE and 
other databases on the NLM computer. About 58% of all survey respondents (63% of 
those who provided the information) obtained their codes in the year after 
GRATEFUL MED was introduced, i.e., during the 12 months ending in July 1987. The 
survey data on methods used to learn to search MEDLINE and to access the database 
indicate that this tremendous 12-month increase in the number of individual code 
holders is due to the combination of the introduction of GRATEFUL MED and the 
continuing effects of the short courses in command language searching. 

Overall, 40% of all respondents (49.7% of those who provided training information) 
indicated that they had taken an NLM sponsored training course and 49% (54.1% of 
those who answered the question) indicated that they always or sometimes used the 
command language. 37% (453% of those who supplied the information) stated that 
they had used GRATEFUL MED as one method for learning to search and 42% 
(47.4% of those who answered the question) indicated that they always or sometimes 
used GRATEFUL MED form screens when searching MEDLJNE. As respondents 
could indicate multiple training methods and some use both the command language 
and GRATEFUL MED form screens to search, there is some overlap in these figures. 
Nonetheless, at the time the survey was taken there were slightly more individuals using 
command language to access NLM's system than were using GRATEFUL MED. 

This situation has changed dramatically since July 1987. Of the more than 3,500 
individuals who have obtained codes since the survey was taken, about 75% indicated 
that they intended to use GRATEFUL MED. In addition to the general increase in 
publicity for the software package, the emphasis in many of the search training courses 
provided by medical librarians has shifted from the command language to the use of 
GRATEFUL MED. In assessing the significance of the survey results, it is important to 
remember that individual command language searchers are now almost certainly a 
minority and that the percentage of individuals using the command language will 
probably continue to decrease. 

While the split between command language and GRATEFUL MED searchers 
represented in the survey results can be viewed as a historical artifact, the data also 
reveal a more basic division of individual users into two different groups that can be 
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expected to persist. The first group is more heavily concentrated in academic settings, 
uses MEDLINE primarily in support of research, and is typically interested in 
comprehensive retrieval. Most of the people in this group are physicians or physician- 
scientists. At the time of the survey, higher percentages of individuals in academic 
settings had taken formal online search training and were using the command language 
'• as compared to individuals in other work settmgs. This is not surprising since to date 
the majority of the special short training classes have been given in academic settings. 

The second group is made up predominantly of physicians who work in private/solo 
practice, group practice, or hospital or clinic settings. These individuals use MEDLINE 
primarily in support of patient care and are more likely (than "research" users) to wish 
to retrieve just a few relevant citations. At the time the survey was conducted, the 
majority of users in private/solo or group practice was already using GRATEFUL 
MED, and a higher percentage of users in hospitals or clinics was using GRATEFUL 
MED as compared to users in academic settings. Individuals in these settings were less 
likely to have access to the short courses in command language searching. 

The lines between the "research" and "patient care" groups blur, particularly in the 
research hospital environment, and it is obvious that the same individual may be a 
"research" user and a "patient care" user at different times. Nevertheless, the survey 
data support the traditional view that there are two basic types of MEDLINE users and 
that they do look for different things from the system. At the time the survey was 
conducted, the "research" group was slighdy larger than the "patient care" group. The 
the latter has much greater potential for growth, however, and can be expected to 
become the predominant group of individual users, if indeed it has not already become 
so. The survey results show that the percentage of new individual users coming from 
academic settings had been declining slowly during the 18 months prior to the survey, as 
the combined percentage of new users from patient care settings was increasing slowly. 

Although more than half of both groups typically retrieve what they perceive as the 
"right number of citations," the survey revealed that retrieval of too few citations is 
perceived as a more frequent problem by "patient care" and GRATEFUL MED users 
than by "research" and command language users. This perception is supported by NLM 
system statistics showing that over 30%> of GRATEFUL MED searches result in no 
retrieval. Analyses of system traffic files of GRATEFUL MED searches indicate that 
non-retrieval most freauently results from combining too many search terms together or 
through misunderstanaing of basic indexing policy (i.e., that the most specific terms are 
assigned by indexers). This is probably due to the fact that fewer GRATEFUL MED 
users receive any formal online search training. The survey findings provide additional 
justification for NLM's efforts to provide some sort of automated assistance to users 
who have retrieved nothing in response to a GRATEFUL MED search. 

Although the "research" and "patient care" groups have some key differences, they also 
share many attributes. Respondents in both groups appear to be relatively young, as 
evidenced by the fact that 67% of all respondents received their highest degree in the 
last 18 years.* Essentially all respondents have access to personal computers and use 

*We compared the age distribution of physician respondents with the age distribution of all U.S. physicians 
as reported in a recent AMA survey (Physician Characteristics and Distribution in the U.S. 1987 edition. 
Chicago, Division of Survey and Data Resources, American Medical Association, 1987, p. 22). For 
purposes of comparison, we assumed that the "average" physician obtains the M.D. degree at age 26. This 
suggests that those who received their MD. degree before 1950 are under- represented among MEDLINE 
access codes holders, while those who received their M.D. degree in the 1960's or 1970*s are 
disproportionately more numerous in the NLM sample than in the U.S. physician population as a whole. 
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them to perform MEDLINE searches. The majority do all their own searches, although 
this amounts to only a few searches in the average month. Most cite greater familiarity 
with the subject or the ability to get search results faster as the principal reason tney 
search for themselves. Overwhelmingly they typically use MEDLINE to satisry 
immediate information needs, rather than to stay current in their fields or to learn 
about new areas. They also typically search for subjects rather than authors or J 0 / 11 ™ 
titles. Overall, they are quite satisfied with MEDLINE, although the majority think tnat 
fewer than half the citations they retrieve are relevant Use of features such as starred 
terms and subheadings can help to exclude irrelevant citations, but analyses or tne 
system traffic files show that these features are rarely used by individual u^ers. 
Additional improvements to GRATEFUL MED and the development of the COACii 
expert search assistant should therefore focus on the problem of refining searches to 
reduce irrelevant retrieval. 

Most respondents are satisfied with the types of information currently available in 
MEDLINE citations, and provided no data on additional information they would like to 
find in the file, even though full-text of articles was one of the choices presented. Of 
the 40% who thought that additional types of information would be useful, nearly two- 
thirds did indicate that the full text of articles would be the single most useful addition. 
Although only a minority of both groups mentioned full text, a greater percentage of the 
"patient care" group is interested in full text than of the "research" group, who tend to be 
in academic settings with better access to the literature. Again, not surprisingly, more 
members of the "research" group than of the "patient care" group listed author's address 
as the single most valuable addition to the MEDLINE record. As of January 1988, the 
principal author's address is carried in MEDLINE records. 

The majority of respondents did indicate some features or capabilities (as opposed to 
information) that would be useful enhancements to MEDLINE. Those respondents 
who specified any desirable improvements generally agree on the principal features or 
capabilities they would like to see added to the NLM system. These are improved 
MEDLINE backfile searching and improved capabilities for printing citations. As 
might be expected, improved print capabilities were of greatest interest to command 
language searchers. Here the survey findings corroborate the opinions of NLM staff. 
Prior to the survey, work had been initiated to improve backfile searching capabilities 
in GRATEFUL MED. GRATEFUL MED users can now indicate at the beginning of 
a search that they wish to search MEDLINE and the backfiles or they can perform a 
backfile search without first searching MEDLINE. These changes were among others 
introduced in Version 3.0 which was mailed to users early in 1988. Also prior to the 
survey, improvements were planned for backfile search capabilities for command 
language searchers. A multi-file search capability was implemented in late 1988. An 
improved print capability was implemented for command language searchers in 
November 1987. 

About 70% of the respondents did not provide any free form comments on the most 
and least satisfactory aspects of MEDLINE. 27% of the respondent group (n=747) 
provided a total of 1,260 comments on the most satisfactory aspects of MEDLINE. The 
pattern of comments was very different for command language searchers and 
GRATEFUL MED users. Command language searchers mentioned the content of 
MEDLINE most frequently, followed in order by the speed and efficiency of the 
system, its hours and availability, and cost. The appreciation for the content of 
MEDLINE is gratifying, considering the resources NLM devotes to selecting the 
contents and ensuring the quality of the database. The MEDLINE database is 
available from a variety of commercial online vendors, and in CD-ROM products, 
however, so it does not represent a particular incentive to use the NLM system. NLM's 
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system also does not differ appreciably from those of other online services in terms of 
speed or accessibility. The widespread availability of short training courses in the use of 
the NLM system and relative cost are probably the factors that led these command 
language searchers to choose NLM's system over other alternatives. 

For GRATEFUL MED searchers who provided comments, convenience appears to be 
primary. GRATEFUL MED itself, which can only be used on NLM's system, is 
mentioned most frequently, followed in rank order by hours and availability, content, 
and speed and efficiency. Cost is mentioned much less frequently by GRATEFUL 
MED users. Older respondents are much more likely to use GRATEFUL MED than 
the command language, another indication that it is attractive to people with relatively 
little previous exposure to computers. These data support the view that GRATEFUL 
MED is making online searching attractive to people who previously thought it too 
inconvenient, time-consuming and cumbersome. 

31% of respondents (n=874) provided a total of 1,218 comments about the least 
satisfactory aspects of the system. Again, there were substantial differences between 
command language searchers and GRATEFUL MED users. The least satisfactory 
aspect mentioned most frequendy by command language searchers was MeSH 
vocabulary /indexing (by 173% of those commenting), followed by command language 
searching itself (16.0%), printing citations (16.0%), and backfile searching (132%), and 
the fact that GRATEFUL MED is currently available for IBM-compatible micros only 
(12.9%). This list matches the Library's view of the principal difficulties with command 
language searching by individuals. Solving problems caused by a lack of understanding 
of MeSH and indexing principles is a major objective of NLM's Unified Medical 
Language System (UMLS) project. GRATEFUL MED was developed to obviate the 
need for health professionals to learn the command, language. As mentioned 
previously, NLM has also taken steps to improve printing and backfile capabilities. A 
Macintosh version of GRATEFUL MED will be available in 1989. 

The small percentage of GRATEFUL MED searchers who provided comments on the 
least satisfactory aspects mentioned specific limitations of the GRATEFUL MED 
package most frequently (37.2%), followed by backfile searching (15.3%), and MeSH 
vocabulary/indexing (10.7%). It should be noted that the survey was conducted before 
the introduction of Version 3.0 of GRATEFUL MED which incorporated many of the 
desired changes. Also, many of the individuals who cited GRATEFUL MED software 
limitations as a least satisfactory aspect also listed GRATEFUL MED as the most 
satisfactory aspect of the system. Thus, it seems that GRATEFUL MED users are 
generally quite satisfied with the software, but would like to have continued 
improvements made in its capabilities. 

The results of this survey provide some objective data on key aspects of individual 
search behavior that have been discussed and debated since the introduction of online 
systems. There has been speculation about whether individuals would stop doing their 
own searching after the novelty wore off. The overwhelming majority of the 
respondents to this survey indicated that their use of MEDLINE had remained the 
same or increased since they obtained their online codes. As most respondents had 
obtained their codes within the year prior to the survey it will be interesting to see 
whether this pattern continues. The likelihood of continued use is increased by the fact 
that the overwhelming majority of survey respondents indicated that they typically 
search for immediate information needs rather than to stay current in their field or to 
learn about new areas. This is the case irrespective of whether respondents use the 
information for research or patient care purposes. It suggests that these individuals are 
integrating at least a modest amount of online searching into their regular work habits. 
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Another topic of discussion is whether the opportunity for direct online searching would 
attract new users or simply alter the mode of access of previous users of search 
intermediary services. The fact that two-thirds of the survey respondents do all their 
own searches suggests that new users are being attracted by the convenience of direct 
online searching. This view is supported by recent evaluations of MEDLINE CD-ROM 
products which also indicate that each new form of access to MEDLINE attracts some 
people who previously did not use the system in any form. 

Controlled vocabularies, and the disparities among them, are often cited as barriers to 
effective use of online systems by individuals users. Over two-thirds of the respondents 
to the survey indicated that they always or usually use MeSH terms when searching for 
subjects in MEDLINE. A similar percentage find MeSH terms very useful or useful. 
The high rate of use of MeSH is a logical consequence of the percentage of respondents 
who had attended at least a 6-hour course in MEDLINE searching and the fact that 
GRATEFUL MED assists the user in locating appropriate MeSH terms. NLM's 
continuing efforts to improve and expand the MeSH cross-reference structure have also 
made the terminology more accessible to those without extensive search training. It 
appears that individual users are willing to use at least a single controlled vocabulary if 
such use is made relatively easy for them. 

There has been considerable debate on whether (and how much) individuals would be 
willing to pay for direct online searching. About two-thirds of the respondents to the 
survey indicated that cost considerations rarely or never keep them from doing a 
MEDLINE search. Less than 10% reported that cost considerations frequently keep 
them from searching. Cost seems to be even less of a consideration for those in group 
practice and private/solo practice than for those in academic and hospital settings. 
These data (plus the continuing growth in the numbers of individual users that NLM 
has experienced since the survey was conducted) indicate that the current cost of 
searching MEDLINE on NLM's system is acceptable to many individuals. 

While the survey provided new and interesting information related to online searching 
by individuals, it also provided objective data to support previously held opinions about 
these users and their views of the NLM system. In particular, NLM staff views about 
the system improvements most desired by individual users are generally corroborated 
by the users themselves. It is encouraging that enhancements recently implemented or 
planned by the Library will address several of these users' concerns with present system 
Features. In general, respondents to the survey appear to have a very positive view of 
NLM's online service. This is evidenced not only by their explicit indications of 
satisfaction with MEDLINE but also by the high return rate of the survey and bv the 
willingness of the large majority of the respondents to participate in follow-up studies. 

The respondents to the survey represent relatively early adopters of direct online 
searching Included in the survey population . are; 1) the very small number of 
individuals who were willing to take the extensive training offered to search 
intermediaries in the earlier days of online retrieval; 2) the much larger eroun who 
were willing to take a 6-hour training course when it began to be offered in 1985- and 
another substantial group who were the first individuals to decide that GRATFFT TT 
MED made direct online searching practical and convenient for them The mnt^,^ 
growth in the number of individual GRATEFUL MED users since the time the I ™!„ 
was conducted suggests that there is a large pool of people who will find direct oniTnl 
searching feasible and attractive when they are made aware of the technoloev nirr^nti 
available. It remains to be seen whether these later adopters of direct online s v 
will differ in significant ways from the users described in this survey searching 
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Key Points 



1. Nearly two-thirds of respondents identified themselves as physicians (65.5%) 
and more than a quarter (27.7%) as scientists. 

o Respondents were instructed to circle all professions that applied and 
10.5% indicated that they were both physicians and scientists. 

2. The majority of individual users of MEDLINE appear to be relatively young. 

o The majority received their highest degree within the past 18 years: 

-40.1% during 1970's 
- 27.1% during 1980's 

3. More than two thirds of respondents (68.6%) indicated they do all searches by 
themselves. 

o On average, they perform 4.3 searches/month by themselves. 

4. Lack of time was the most frequent reason given for having someone else 
perform MEDLINE searches (by 593% of those who have others search for them) 

5. The two main reasons given for performing their own searches were: 
o greater familiarity with subject matter (47.9%) 

o ability to get search results faster (32.2%) 

6. Among all reasons selected (regardless of rank assigned), two-thirds of the 
respondents {655%) indicated enjoyment of searching as a factor. 

The majority of respondents have had their codes for two years or less (90.1%). 
The majority of the respondents also do not share their code with anyone else (82%). 

8. The majority of respondents (84.5%) feel they are somewhat experienced or not 
very experienced in the use of online databases. 

9. For over half of all respondents (57.7%), online usage has remained about the 
same since receiving their access codes. 

o 30% indicated usage has increased 

o 123% indicated usage has decreased 

10. Cost seems not to be of overwhelming importance in individuals' use of MEDLINE. 
The majority (67.6%) indicated that cost considerations rarely (26.6%) or never 
(41.0%) keep them from performing a search. 

11. The overwhelming majority (96.0%) say they most often search for a subject. 
81.7% indicated they typically search to satisfy an immediate information need. 
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12. Those whose primary use of MEDLINE search information is in research/testing 
are more likely to want all relevant citations (67.1%) compared to those whose primary 
use of MEDLINE search information is in education (53.3%) or patient care (49.8%). 

13. Those in group and private/solo practice are somewhat more likely to feel that less 
than half of the citations they typically retrieve are relevant. (63.4% of those in group 
practice and 61.1% of those m private/solo practice compared to 58.6% of all 
respondents.) 

14. Almost two-thirds of all respondents (60.8%) indicated MEDLINE citations are 
acceptable in their current form (i.e., answered "NO" to additional information needed 
in a citation). 

o of the 39.2% who said additional types of information would be valuable, 
full text was overwhelmingly chosen as the single most valuable piece of 
information not presently available in MEDLINE citations. 



15. Eighty percent of respondents expressed an overall satisfaction with MEDLINE. 

o Individuals also are satisfied with their chosen method of accessing 
MEDLINE 

16. Two-thirds of the respondents (64.1%) took less than 10 minutes on a typical 
search. 

o majority of respondents (83.4%) feel that length of time to conduct a 
search on MEDLINE is reasonable. 

17. Of the group of individuals who indicated research/testing as one of their uses of 
MEDLINE information, 61.5% ranked it as their most common use. Of those who 
indicated patient care, 52.8% ranked it as the most common use. 

18. A somewhat larger portion of the physician group (30.1%) always use GRATEFUL 
MED form screens as compared to the other professional groups (other health 
professional 24.8%, scientist 21%, nurse 18.5% and student 17.1%). 

o one third of respondents in private/solo and group practice always use 
GRATEFUL MED form screens to search. 



TABLE 1 



PROFESSION OF RESPONDENTS 
N = 2710 





Frequency 


Percent* 


Physician 


1776 


65. 5% 


Nurse 


99 


3.7% 


Other Health Professional 


180 


6.6% 


Scientist 


753 


27.8% 


Student 


118 


4.4% 


Librarian/ Info Specialist 


86 


3.2% 


Other* 


207 


7.6% 


Total 


3219 





No answer: 6 



*Although 2710 individuals answered this questions, some indicated more 
than one profession; therefore, the frequency column is greater than 
2710 and the percent column exceeds 100%. 



Table 2 

Specialty of Survey Respondent 



N = 2197* 



Specialty of Respondent 



Internal Medicine 

Medical Oncology 

Pathology 

Pediatrics 

Family Practice 

Surgery 

Psychiatry 

Cardiovascular Disease 

Hematology 

Anes thes io logy 

Radiology 

Dermatology 

Neurology and/or Child Neurol. 
Pulmonary Disease 
Obstetrics and Gynecology 
Gastroenterology 
Ophthalmology 

Endocrinology and Metabolism 
Infectious Disease 
Orthopaedic Surgery 
Neurological Surgery 
Emergency Medicine 
Nephrology 

Psychiatry and Neurology 

Allergy 

Urology 

Rheumatology 

Pediatric Hematology-Oncology 

Pediatric Endocrinology 

Physical Medicine and Rehab. 

Neonatal-Perinatal Medicine 

Pediatric Nephrology 

Nuclear Medicine 

Otolaryngology 

Plastic Surgery 

Preventive Medicine 

Thoracic Surgery 

Pediatric Cardiology 

Colon and Rectal Surgery 

Blood Banking 

Child Psychiatry 

Other 

Total 



Frequency 



282 
147 
125 
109 
99 
95 
85 
75 
64 
58 
56 
53 
53 
51 
50 
44 
44 
40 
39 
35 
29 
25 
22 
21 
20 
18 
16 
13 
11 
10 
8 
8 
7 
7 
7 
6 
6 
5 
4 
3 
1 

469 



% of 
Cases 

12.8 
6.7 
5.7 
5.0 
4.5 
4.3 
3.9 
3.4 
2.9 
2.6 
2.5 
2.4 
2.4 
2.3 
2.3 
2.0 
2.0 
1.8 
1.8 
1.6 
1.3 
1.1 
1.0 
1.0 
0.9 
0.8 
0.7 
0.6 
0.5 
0.5 
0.4 
0.4 
0.3 
0.3 
0.3 
0.3 
0.3 
0.2 
0.2 
0.1 
0.0 
21.3 



2320 
572 



No answer: 

*Some respondents indicated more than one specialty; therefore, 
precent exceeds 100 and frequency exceeds N of 2197. 



TABLE 3 
HIGHEST DEGREE RECEIVED 

N=2733** 

% of 





Frequency* 


Cases 


M.D. 


1760 


64.4 


Ph.D. 


570 


20.9 


Masters 


222 


8.1 


Bachelor ' s 


149 


5.5 


Phann.D. 


39 


1.4 


Nursing 


33 


1.2 


Dentistry 


30 


1.1 


D.O. 


23 


.8 


Other Doctorate 


21 


.7 


Other 


17 


.6 


DVM 


15 


.5 


High School 


9 


.3 



* This question allowed for multiple responses. 

**The total number of respondents exceeds 2716 because this question 
was answered by some individuals who were later determined to be 
ineligible and were then excluded from further computations. 



TABLE 4 

YEAR RECEIVED HIGHEST DEGREE 
N=2667 

Cumulative 

Frequency Percent Percent 



Pre-1950 83 3.1 3.1 

1950s 270 10.1 13.2 

1960s 523 19.6 32.8 

1970s 1069 40.1 72.9 

1980s 722 27.1 100.0 

Total 2667 

No answer: 49 
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TABLE 6 

PRIMARY WORKPLACE OF RESPONDENTS 
N=2701 



Private/Solo 
Practice 

Group Practice 

Hospital/Clinic 

College/Univ. 
Medical School 

Private Company/ 
Business 

Government Agency 

No Formal Workplace 

Other Workplace 



Frequency 
340 

331 
395 
1264 



89 

145 
28 
109 



Percent 
12.6 

12.3 
14.6 
46.8 

3.3 

5.4 
1.0 

4.0 



Cumulative 
Percent 

12.6 



24.8 
39.4 
86.3 

89.6 

94.9 
96.0 
100.0 



No answer: 



15 
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TABLE 8 

# OF SEARCHES PERFORMED BY RESPONDENTS 
IN A TYPICAL MONTH 

N-2686 

Cumulative 

Frequency Percent Percent 



None 323 12.0 12.0 

One 570 21.2 33.2 

Two 523 19.5 52.7 

Three 355 13.2 65.9 

Four 249 9.3 75.2 

Five 242 9.0 84.2 

Six or more 424 15.8 100.0 

Total 2686 

No answer: 30 



TABLE 9 

// OF SEARCHES PERFORMED BY OTHERS 
IN A TYPICAL MONTH 

N-2682 



None 

One 

Two 

Three 

Four 

Five 

Six or more 



Frequency 
1842 
405 
197 
78 
54 
55 
55 



Percent 
68.6 
15.1 
7.3 
2.9 
2.0 
2.0 
2.0 



Cumulative 
Percent 

68.6 

83.7 

91.0 

93.9 

95.9 

98.0 

100.0 



Total 



2682 



No answer: 



30 



Librarian/ Info . 
Specialist 

S tudent / Research 
Assistant 

Secretary/ 
Admin. Assist. 

Colleague 

Family Member 

Other 



TABLE 10-A 
OTHERS WHO PERFORM SEARCHES 
N=1103 



Frequency 
903 

58 

AO 

51 
16 
35 



Percent 
81.9 

5.3 

3.6 

A. 6 
1.5 
3.2 



Cumulative 
Percent 

81.9 

87.1 

90.8 

95.4 
96.8 
100.0 



Total 



No answer; 



1103 



1613* 



*Note that not all respondents were eligible to answer this question 
since many never used a search intermediary. 
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TABLE 11 -A 
REASONS FOR HAVING OTHERS SEARCH MEDLINE 
N=1151 

FREQUENCY PERCENT * 

When someone else can do it 220 19.1% 

as easily as I can 

When I don't have time to 682 59.3% 

do it myself 

When I need different expertise/ 445 38.7% 

system knowledge 

When my search results have 317 27.5% 

not been satisfactory 

Other 137 11.9% 

Total 1801 
No answer 1565 



"'Respondents were instructed to circle all reasons that applied; 
therefore, percentages total to more than 100% and frequencies total 
more than the N of 1151. 
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2 

Neutral 
3 
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TABLE 12 

SATISFACTION WITH SEARCHES DONE BY OTHERS 
N=1121 



Frequency 

383 
408 

230 

86 
14 



Percent 

34.2 
36.4 

20.5 

7.7 
1.2 



Cumulative 
Percent 



34.2 
70.6 

91.1 

98.8 
100.0 



Total 



1121 



No answer: 



1595* 



*Note that not all respondents were eligible to answer this question, 
since many never used a search intermediary. 



TABLE 13 



REASONS FOR DISSATISFACTION 
WITH SEARCHES DONE BY OTHERS 



N=150 



Inconvenient location 
Inconvenient hours 
Have to wait for search 
Cost 

Unsatisfactory results 
Other 



FREQUENCY PERCENT * 

17 11.3% 

24 16.0% 

59 39.3% 

36 24.0% 

93 62.0% 

46 30.7% 



Total 150 



No answer 2566 



^Respondents were instructed to circle all reasons that applied; 
therefore, percentages exceed 100% and frequencies are greater than the 
N of 150. Note that not all respondents were eligible to answer this 
question since many never used a search intermediary. 
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TABLE 16 
EXPERIENCE WITH ONLINE DATABASES 
N=2561 



Very Experienced 
Somewhat Experienced 
Not Very Experienced 
Not Experienced 



Frequency 
212 
1244 
920 
185 



Percent 
8.3 
48.6 
35.9 
7.2 



Cumulative 
Percent 

8.3 

56.9 

92.8 

100.0 



Total 



2561 



No answer: 



155 
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TABLE 18 

LENGTH OF TIME HAVE HAD MEDLINE CODE 
N=2509 



6 Months or less 

7 to 12 Months 
13 to 18 Months 
19 to 24 Months 
25 to 30 Months 
31 to 36 Months 
37 to 42 Months 
More than 42 Months 

Total 

No answer: 



Frequency 
808 
776 
212 
464 

47 
114 

10 

78 

2509 
264 



Cumulative 



Percent 


Percent 


32.2 


32.2 


31.0 


63.1 


8.4 


71.6 


18.5 


90.1 


1.9 


91.9 


4.6 


96.5 


.4 


96.9 


2.7 
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Three to Five 
Six or more 



TABLE 20 

NUMBER OF PEOPLE SHARING MEDLINE CODE* 
H-2661 



Frequency 
17 
2183 
229 
167 
65 



Percent 
.6 
82.0 
8.6 
6.3 
2.4 



Cumulative 
Percent 

.6 

82.7 

91.3 

97.6 

100.0 



Total 



No answer: 



2661 
55 



^Respondents, were instructed to write "1" if they were the only person 
to use the code. 



Usage Has: 
Increased 
Stayed the Same 
Decreased 

Total 



TABLE 21 
USAGE OF MEDLINE 
N=2545 



Frequency 
763 
1469 
313 

2545 



Percent 
30.0 
57.7 
12.3 



Cumulative 
Percent 

30.0 

87.7 

100.0 



No answer: 



171 



TABLE 22-A 

REASONS GIVEN FOR INCREASE IN USE OF MEDLINE 

BY 

WHETHER USE HAS INCREASED OR DECREASED 
N=678* 

Frequency Percent 



Lack of time 2 .37. 

Increases productivity 124 18. 3% 

The more I learn, the 25 3.77. 
more I use it 

Search for other people 18 2.7% 

Enjoy using 13 1.9% 

Increased familiarity 292 43,1% 

Increased work load/need 169 24.9% 

Money reasons 8 1.2% 

Other services unavailable 9 1.3% 

Better than others 12 1.8% 

Change in research needs 6 

Change from different systems 4 .6% 

GRATEFUL MED 26 3.8% 

Saves time (easier to use) 11 1.6% 

Better computer hardware 3 .4% 



Total 722 



*0f the 763 individuals who indicated that their usage had increased 
678 indicated reason(s) why this was so. Content of respondent 
answers was sometimes coded with more than one code; therefore, total 
% exceeds 100% and frequencies are greater than the N of 678. 



TABLE 22-B 

REASONS GIVEN FOR DECREASE IN USE OF MEDLINE 

BY 

WHETHER USE HAS INCREASED OR DECREASED 
N-288 



Frequency Percent 



Lack of time 


77 


26. 7% 


Decreased work load/need 


71 


24.7% 


Money reasons 


30 


10.4% 


Don't like using 


25 


8.7% 


Confused when using 


28 


9.7% 


Like others better 


10 


3.5% 


Use other services 


19 


6.6% 


Change in research needs 


10 


3.5% 


GRATEFUL MED 


3 


1.0% 


Hardware inoperable (bad 
telephone lines) 


15 


5.2% 


Forget that it exists 


1 


.3% 


Limited access to computer 


2 


.7% 



Total 291 



*Of the 313 individuals who indicated that their usage had decreased, 
288 indicated reason(s) why this was so. Content of respondent answers 
was coded with more than one code in a few instances; therefore, total 
% slightly exceeds 100% and frequencies are greater than the N of 288. 



TABLE 23 

EFFECT OF COST CONSIDERATIONS ON SEARCHING 
N-2554 

Cumulative 





Frequency 


Percent 


Percent 


Never a consideration 


1047 


41.0 


41.0 


Rarely a consideration 


680 


26.6 


67.6 


Occasionally a consideration 


591 


23.1 


90.8 


Frequently a consideration 


236 


9.2 


100.0 


Total 


2554 
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TABLE 25 



Immediate 
Information 

Stay Current 

Learn New Areas 

Other 



REASONS FOR WHICH INDIVIDUALS 
TYPICALLY SEARCH 

N=2556 



Frequency 
2087 

263 
123 
83 



Percent 
81.7 

10.3 
4.8 
3.2 



Cumulative 
Percent 

81.7 



91.9 
96.8 
100.0 



Total 



2556 



No answer: 



160 



2£ 

a 

u 

llrtO 

.c a-4 
o n c 

O w 

3 



O r>. 
oo • 
o — < 
cm oo 



CM cm 

in 
oo 



cm cn 
vO • 

CM O 



m >o 



CM 00 
CM • 



CM ON 



cn m 
oo • 



on cn 



r- o 

<T • 

m o 

CM O 



00 o 
o • 

-H O 

o 



CO 

o a<M 

U C 
O O w 
Z 3 



~* ao 



cm r-* 



K 

vO O 
CM • 
O 

o 



- G <T 

> a> — • 

O 00 H 

CJ < c 



CM 



CM 00 



ON VO 



cm m 



O 



o 



— . 01 

U CO ON 
C ao 
> -H N 
-« tn c 
U 3 w 

a, as 



he 

00 CM 
00 



in cm 



cn 



r*. o 
ao • 
o 
o 



4 o m 

4J ^ ON 

c cn 

CLth U 

o) ^ c 
O O w 



K 

O • 

cn cm 

00 



ON ON 
CM • 



O -tf 
CM • 

m 



ao o 

NO • 

CO O 

o 



> O 

o 

C J= 

' — C/l 

O CO 



ON O* 



O O 

m • 

CM 



ON 00 

vO • 

m 



— NO 



<T O 

ON • 

•H O 

— • o 



CO ^ 

CJ ^ 
O. ft d 

3 4J cn 
0 O II 
h it C 

0m 



CM 00 
00 



cn «r 

CM • 



SO ON 



cn 



ON O 

o • 
cn o 
o 



CM 
CM 



o 

»H CO 
0 CJ 
C/J -H 
4J 

> o 

•H cd 

u u 

04 a. 



O vO 

NO • 

cm cn 
ao 

c 
o 

CD *J 
*j CO 

^ i 

TJ o 

CO 

g c 



sr m 
cm • 



4J 

c 

0) 

u 
u 
3 
U 













cn 


«— ( 


m 












m 




cn 










<u 








z 










01 






c 


CO 


Vi 




u 


eo 


CO 




<fl 


u 


J= 




co 


< 


4J 




-j 




o 





00 o 



cn o 
o 



CM 
CM 



U 

a) 

w 
c 

<0 

o 
z 



TABLE 27 

# OF CITATIONS USUALLY INTERESTED IN RETRIEVING 

N=2550 



A Few citations 

All Relevant 
Citations 

Other 



Frequency 
973 
1503 

74 



Percent 
38.2 
58.9 

2.9 



Cumulative 
Percent 

38.2 

97.1 

100.0 



Total 



2550 



No answer: 



166 



00 M 

(N PU 



(0 O 

3 4J OJ 

o o u 

a e- a. 



C 

O K 



V) 

O 

a 

cn u 
< Ph 







K 


K i 














cn 


O 


O l 




K 




on 




ON 


r>. 


CO 


O 1 






00 


00 


CM 


cn 


m 




O I 






CO 


m 






















CM 


CM 


vO 


O I 




M 


in 


«-4 


on 






m 


r» i 




ca a 


00 


m 






o 




00 1 


3 


4J 0) 


to 


o 






— * 




~* i 


o 


O M 
















05 


H Cn 









i 




M 


K 


K 




1UIU1 




K 


K 






«n 


m 


O 


O l 


o 


JM 




in 


<r 


cj 












a 














r*. 


00 




O 






O 


vO 


cn 


*-> 




CM 


so 




O i 


4J 




cn 






W) 










| 


■ 










o 






















6 












i 














CM 


-a- 


CM 


00 I 






<— • 


— < 




J- 




00 


o 




on I 


u 




sr 






cn 


a 




CM 




CM 1 


cn 


a 




cn 






a> 












9) 










u 










0) 


U 








< 


Cn 










«< 


Cn 









u 


O 




K 


K 


M 


M 


e 


o 




a 


o 


SM 


m 


CM 


CM 


O 


o 


o 




u 














■H 








4J 




CM 




cn 


O 


4J 


4J 




4J 


cn 




cn 


SO 




o 


4 


cn 




c 


o 










■— t 


a 


o 




0 


S 












3 


E 




-H 














T3 






4J 














Ed 


•o 




03 


c 




t— • 




on 






C 




0- 


CM 


a 


ON 






00 




CM 


a 






3J 




cn 




m 






OJ 




01 


U 












0) 


u 




< 


En 












< 


Cn 





K 




K 


vO 


00 


m 


O 










r>- 


ON 


CM 


o 


-» 






o 










ON 




m 


m 




ON 


CM 


00 


-ST 






ON 



c 












-a 






a) 






4J 






t/> 






(V 




cn 


u 




c 


<u 




o 


*J 




1-t 


c 




4J 


1-1 




(0 




00 


4-1 


>> 


c 


-1 






u 




> 




08 


1-1 




O 


<u 


OJ 


-<•* 


CJ 


Cn 


Q. 


a) 






M 


< 









c 

> 
a 
i— i 
ai 

06 U 
0) 



e 

O K 



O 
6 ol 
a) 

(0 u 
< Cn 



o 



Sv" 

o 



o 
a 



00 





K 


IM> 




o 




ON 


o 


— « 






o 




m 




o 










ON 


o 


On 


00 


o 


• r>* 




ON 




m 




ON 








ts? 


cn 


cn 


<• 


O 


CM 


cn 




d 


ST 


m 




o 








» 


in 


o 




ON 


cn 






t 








cn 



C 




(0 








on 




•o 




i-t 




0) 




4J 




4J 




«0 




cn 




4J 




tt 




i-t 




u 


c 


u 




0) 


o 






4-1 


iH 


4J 




c 


4-> 


c 












00 


4-) 


> 




>> c 


i-t 








o 


r-t 




^ > 




ai 




CTJ i-l 




OS 


u 


U 0) 


0) 




a) 


i-l U 


Cn 


i— t 


JZ 


a. ai 






4J 


>. u 


< 


< 


O 


H 













ft* 


r-» 


-3" 


00 








co 


co 


CN 


co 


NO 






o 






cm 


m 




CM 






»H 





t* 
o 



o 
o 



co 
cm 



o 

u 

4-1 
(A 

o 
a 

T3 

U 

co a 
ai 

(0 u 



Q 



W 

O 
6 

T3 
C 



a 

0) 
(A P 



C 

O 6* 



O 

a 
*-> 

i 



ol 











© 










in 


CM 


CM 




m 






en 


00 


in 






H 






6< 


6* 




m 


co 


CM 








co 


CM 




co 


o 




vO 


en 




co 


in 




< 


CM 







8* 


t* 


in 


.— t 










o 




CM 


co 


nO 




1-4 


-4 


ON 


o 


ON 


CM 


co 


r*. 





o 



o 
o 



CO 
CO 
CO 



o 



o 
o 



NO 

o 



6* 

o 



o 
o 



00 



u 
a) 

X 

■u 
o 
c 

CO 

a) 
c 

o 



4-> 

c 
a> 
-a 
c 
a) 
a. 
<u 

T3 

c 



T3 

0) 

N 

>> 
r-t 

CO 

c 

CO 

o 
CO 

a) 

0) 
cd 
00 

c 

•H 
4J 
W 

a) 
o 

M 
CO 

a) 
en 
a) 

OS 

T3 

C 
CO 



c 




•H 




T3 




<D 




4J 




W 




a) 




t~ 




<u 




u 




c 




i-l 






00 


>» 


c 


fH 


-H 


»H 


> 


(0 


•H 


O 


0) 


fH 


u 


a. 


<u 


>> 




H 







0] 






c 






o 






•H 






4-1 






CO 






4-i 




w 


i-l 




c 


o 




o 








4-1 




■u 


c 




CO 


CO 




4J 


> 




ft 


o 




o 


e l 






OS 




CD 




a) 




fH 






4-> 


«< 


< 


o 



c 

o 
ft 

4J 

CO 

o 
3 
T3 
W 



a) 

CO 
U 



CO 
-uJ 

o 



0- 



I 



0) 

u <o on 

w ^ ii 
one 

O w 

3 



ON ^ 

vO • 
ON CO 

m 



r-i oo 
Ox 

m 



oo o 

ON • 

*T ON 



00 f>> 

cn . 



<T ON 



-* O 

m o 

CN o 



00 o 
o • 
~* o 
o 



0) 

3 u 
6 n ^ 

S ^ 00 
O Q. 04 

Oh U 
^ C 
0 O w 

Z 3: 



O <n 



in r>» 



— • 00 



o o 



o 
o 



- c 

> CD -« 
0 00 II 



CN O 

m • 
oo 
m 



ON 

to 



-T ON 



m . 
o 
o 



— CO 

O tfl --s 
C_> CD ON 
C 00 

> H 
•»* W C 

U 3 w 
0* 03 



oo r» 



NO ON 
<• • 
CN 

m 



O 



o 
o 



H U i/1 

•U -rt ON 

C cn 

a. ii 

CO i-* c 

o u w 
SB 



— ON 

CO 



O • 
CN U-l 

m 



ao on 



ao o 

vO • 
cn O 

o 



> O 

O <T 

C -C NO 

3 y n 

Cfl — < 

^ n 

h e 

O CD w 



SO 

00 • 

m 



00 CN 



r»» in 
vO 



on <r 



m o 
on • 
— • o 

-4 o 



CO ^— s 

o 

a. -« m 

3 -U PO 

o u ii 

u Cfl C 

Oh 



m m 
m • 
o 

m 



NO NO 



NO O 



O • 
CO O 
O 



CN 



o 

t-t a) 

QUO 

tsi ii <: 

-u n 

> O II 

"H CO C 

l-i U w 
CU Oh 





















ON 




o 


O 


vO 


ON 


tn 


o 


m 














-H 




CN 




«cr 










n 


d 










«n 








o 


















— < 


























c 
















CO 


<o 


co 














c 


> 


c 










u 




o 




o 










cu 






i-l 


1-1 














4-1 


CD 


4J 








CO 


CO 


3 


CO 


OS 




u 




t-i 


i-H 


C 


<u 


•u 






cu 






CO 


CO 




-H 






£ 






4J 






o 




u 






o 


o 


o 


< 




< 




O 




u 


H 


z 



TABLE 30 

NUMBER OF CITATIONS TYPICALLY RETRIEVED 
N-2510 



Frequency 
Too Few Citations 385 
The Right Number 1533 
Too Many Citations 592 



Valid 
Percent 

15.3 

61.1 

23.6 



Cumulative 
Percent 

15.3 

76.4 

100.0 



Total 



2510 



No answer: 



206 
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TABLE 32 

METHOD OF ACCESS 
BY 

NUMBER OF CITATIONS TYPICALLY RECEIVED 



Access Too Few me nigu«- Trt1 - a i 

Method Citations Number Citations Total 

Freq % Freg. Z Frea " 



Too Few TheMght TooJUn^ Hoj^ 

Method 
Always use: 

Direct Co-nnand 107 10.2 676 64.8 261 25.0 1044 100.0 

Language 

GM For. 153 23.6 369 56.9 127 19.6 649 100.0 

Screens 



TABLE 33 

PERCENTAGE OF CITATIONS RELEVANT 
TO SEARCH QUERY 

N=2399 

Cumulative 

Frequency Percent Percent 

0% to 25% 566 23.6 23.6 

26% to 50% 841 35.1 58.6 

51% to 75% 541 22.6 81.2 

76% to 100% 451 18.8 100.0 

Total 2399 
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TABLE 35 
TYPE OF SEARCH TYPICALLY DONE 
N=2552 



An Author 

A Journal Title 

A Subject 

Total 



Frequency 
81 
22 
2449 

2552 



Percent 
3.2 
.9 
96.0 



Cumulative 
Percent 

3.2 

4.0 

100.0 



No answer: 
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TABLE 36-A 

AREAS IN WHICH SEARCH INFORMATION IS USED 
BY INDIVIDUALS 
(Includes ranking of 1, 2, or 3) 

N=2716 



Patient Care 

Education 

Research/Testing 

Management/ 

Administration 

Regulation 

Other 



Frequency 
1877 
1793 
1929 
201 

79 
110 



Percent 
69.1% 
66.0% 
71.0% 
7.4% 

2.9% 
4.0% 



TABLE 36-B 



MOST COMMON AREA FOR WHICH SEARCH INFORMATION 
IS USED BY INDIVIDUALS 



Patient Care 

Education 

Research/Testing 

Management/ 

Administration 

Regulation 

Other 



N=2592 

Frequency 
991 
321 
1186 
30 

9 
55 



Percent 
38.2% 
12.4% 
45.8% 
1.2% 

.3% 
2.1% 



Total 



2592 



100.0% 
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TABLE 39-A 
USE 0? MESH 
N-2544 



Always 

Usually 

Occasionally 

Rarely 

Never 



Frequency 
612 
1109 
470 
224 
129 



Percent 
24.1 
43.6 
18.5 
8.8 
5.1 



Cumulative 
Percent 

24.1 

67.6 

86.1 

94.9 

100.0 



Total 



2544 



No answer: 



172 



TABLE 39-B 
PERCEIVED USEFULNESS OF MESH 
N-2475 



Very Useful 

Useful 

Neutral 

Not Useful 

Not at all useful 



Frequency 
662 
966 
575 
192 
80 



Percent 
26.7 
39.0 
23.2 
7.8 
3.2 



Cumulative 
Percent 

26.7 

65.8 

89.0 

96.8 

100.0 



Total 



2475 



No answer: 



241 
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TABLE 41 



REASONS WHY 


INDIVIDUALS DO NOT 


USE MESH 




N=454 






Frequency 


Percent* 


Unfamiliar with MeSH 


116 


25.6% 


Cumbersome terms 


107 


23. 6% 


Terms too specific 


6 


1.3% 


Terms too general 


49 


10.8% 


No terms for my area 


57 


12.6% 


Inappropriate retrieval 


11 


2.4% 


Lose relevant citations 


1 A 


i l ▼ 

J . l/» 


Indexing inconsistency 


7 




Text word is easier/better 


48 


10.6% 


No manual/don't want to use 


45 


9.9% 


Other 


59 


13.0% 


Total 


519 





^Content of respondent answers was somtimes coded with more than one 
code; therefore, percentages exceed 100% and frequencies exceed the N 
of 454. Percentages used are based on the number of individuals 
responding to this question (454). 
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TABLE A3 

SATISFACTION WITH SEARCHES DONE BY SELF 
N-1318 

Cumulative 

Frequency Percent Percent 

Satisfied 

1 358 27.2 27.2 

2 597 45. 3 72.5 
Neutral 

3 257 19.5 92.0 
Dissatisfied 

* 77 5.8 97.8 

5 29 2.2 100.0 



Total 

No answer: 



1318 
1398 
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TABLE 45 

OVERALL SATISFACTION WITH MEDLINE 
N-2539 

Cumulative 

Frequency Percent Percent 



Satisfied 

1 773 30. A 30.4 

2 1293 50.9 81.4 
Neutral 

3 367 14.5 95.8 
Dissatisfied 

4 90 3.5 99.4 

5 16 .6 100.0 

Total 2539 

No answer: 177 
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TABLE 54-A 
EQUIPMENT USED WHEN SEARCHING 
N-2556 

Cumulative 

Frequency Percent Percent 

A PC 2454 96.0 96.0 

A Terminal 102 4.0 100.0 

Total 2556 



No answer: 



160 



TABLE 54-B 
PC AVAILABLE AT WORKPLACE 
N-2708 



Cumulative 

Frequency Percent Percent 

Yes 2600 96.0 96.0 

No 108 4.0 100.0 

Total 2708 

No answer: 8 
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TABLE 59 



PROBLEMS ACCESSING MEDLINE 
ON THE NLM COMPUTER 

N-2392 

FREQUENCY 



PERCENT* 



No problems 

Remembering the connect/ 
disconnect protocols 

Busy telecommunication 
lines 

NLM computer not available 
Other 



1287 
386 

466 

281 
252 



53.8% 
16. 1% 

19. 5% 

11.7Z 
10.5% 



Total 



2672 



No answer: 



324 



^Respondents were asked to circle all reasons that applied; therefore, 
percentages exceed 100% and frequency exceeds N of 2392. 
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TABLE 61-A 

SATISFACTION WITH 3-5 DAY INITIAL TRAINING COURSE 

N=328* 

Cumulative 

Frequency Percent Percent 



Satisfied 

1 170 51.8 51.8 

2 105 32.0 83.8 
Neutral 

3 AA 13. A 97.3 
Dissatisfied 

A 6 1.8 99.1 

5 3 .9 100.0 

Total 328 

No answer: 2388 



*0f the 1095 individuals who indicated they had received some form 
NLM sponsored training, 328 had taken a 3-5 day training course. 
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TABLE 62 -A 

SATISFACTION WITH 6 HOUR BASICS OF SEARCHING MEDLINE COURSE 

N-1040* 

Cumulative 

Frequency Percent Percent 

Satisfied 

1 464 44.6 44.6 

2 404 38.8 83.5 
Neutral 

3 133 12.8 96.3 
Dissatisfied 

4 31 3.0 99.2 

5 8 .8 100.0 

Total 1040 
No answer: 1676 



*0f the 1095 individuals who indicated they had received some form of 
NLM sponsored training, 1040 had taken the 6 hour Basics of Searching 
MEDLINE course. 
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TABLE 65 

MOST SATISFACTORY ASPECTS OF MEDLINE 
N=747 





Frequency 


Percent* 


Great service/system 


64 


8.6% 


Good content/depth/ information 


244 


32.7% 


GRATEFUL MED 


149 


19.9% 


Cost reasonable 


140 


18.7% 


Availability/Hours/ Convenience 


190 


25.4% 


Speed/ efficiency/ time-saving 


184 


24.6% 


Ease of use/user-friendly 


86 


11.5% 


MeSH vocabulary/ indexing 


52 


7 . 0% 


Other NLM databases available 


14 


1.9% 


Training courses 


14 


1.9% 


Service desk/assistance 


47 


6.3% 


Documentation/ manuals 


18 


2.4% 


Other 


58 


7.7% 


Total 


1260 




No answer 


1969 





*Content of respondent answers was frequently coded with more than one 
code; therefore, percentages exceed 100% and frequencies exceed the N 
of 747. 



TABLE 66 

MOST SATISFACTORY ASPECTS OF MEDLINE 
BY 

ACCESS METHOD USED TO SEARCH MEDLINE 



Method of access 



Always Use: 



Direct Command Language 
N-305 

Frequency Percent* 



Great service/ system 27 

Good content /depth/ info. 121 

GRATEFUL MED 3 

Cost 75 

Availability/hours/ 84 
convenience 

Speed/ef f iciency/ 89 
time-saving 

Ease of use/user-friendly 26 

MeSH vocabulary/ indexing 17 

Other NLM databases 6 
available 

Training courses 7 

Service Desk/assistance 13 

Documentation/manuals 11 

Other 28 



8.9% 
39. 8% 

1.0% 
24.7% 
27. 6% 

29. 3% 

8.6% 
5.6% 
2.0% 

2.3% 
4.3% 
3.6% 
9.2% 



Frequency 
13 
45 
89 
25 
49 

42 

29 
17 



GM Form Screens 
N-194 
Percent* 



15 

1 
8 



6.7% 
23.2% 
45.9% 
12.9% 
25.3% 

21.6% 

14.9% 
8.8% 



7.7% 
.5% 
4.1% 



Total 



507 



333 



*Content of respondent answers was frequently coded with more than one 
code; therefore, percentages exceed 100% and frequencies exceed the N's 
of 305 and 194. 



TABLE 67 

LEAST SATISFACTORY ASPECTS OF MEDLINE 
N=874 







Percent* 


Backfile searching 


ILL 


14. 0% 


Direct/ command language searching 




11.0% 


Printing [selective] 


Q t\ 


1 0 2% 


Other NLM system limitations 


oJ 


9.5% 


Full text not available 


0 J 


7.2% 


No abstracts for some citations 


14 


1.6% 


Poor coverage in area 


48 


5.5% 


MeSH vocabulary/ indexing 


120 


13.7% 


Cost 


50 


5.7% 


GRATEFUL MED limitations 


150 


17.2% 


GRATEFUL MED not available 
for all microcomputers 


68 


7.8% 


Need for more training 


29 


3.3% 


Need less technical newsletter 


*i 
j 


.3% 


Documentation/manuals 


AO 


A. 6% 


NTIS billing system 


29 


3.3% 


User hardware/ software problems 


25 


2.9% 


Service desk/assistance 


6 


.7% 


Other NLM databases 


12 


1.4% 


Other 


171 


19.6% 


Total 


1218 





^Content of respondent answers was frequently coded with more than one 
code; therefore, percentages exceed 100% and frequencies exceed the N 
of 874. 



TABLE 68 

LEAST SATISFACTORY ASPECTS OF MEDLINE 
BY 

ACCESS METHOD USED TO SEARCH MEDLINE 



Method of access 



Always Use: Direct Command Language GM Form Screens 

N-387 N»196 
Frequency Percent* Frequency Percent* 

Backfile searching 51 13. 2% 30 15. 3% 

Direct/ command language 62 16. 0% 3 1.5% 
searching 

Printing [selective] 62 16. 07. 8 4.1% 

Other NLM system 49 12.7% 3 1.5% 
limitations 

Full text not available 18 4.7% 16 8.2% • 

No abstracts for some 6 1.6% 2 1.0% 
citations 

Poor coverage in area 21 5.4% 11 5.6% 

MeSH vocabulary/ indexing 67 17.3% 21 10.7% 

Cost 22 5.7% 9 4.6% 

GRATEFUL MED limitations 10 2.6% 73 37.2% 

GRATEFUL MED not available 50 12.9% 4 2.0% 
for all microcomputers 

Need for more training 10 2.6% 6 3.1% 

Need less technical 3 .8% 
newsletter 

Documentation/manuals 23 5.9% 4 2.0% 

NTIS billing system 20 5.2% 1 .5% 

User hardware/ software 9 2.3% 4 2.0% 
problems 

(continued on next page) 



TABLE 68 (cont.) 

LEAST SATISFACTORY ASPECTS OF MEDLINE 
BY 

ACCESS METHOD USED TO SEARCH MEDLINE 



Method of access 



Always Use: 

Service Desk/Assistance 
Other NLM databases 
Other 
Total 



Direct Command Language 
N-387 

Frequency Percent* 



1 
5 
68 
557 



.3% 
1.3% 
17. 6% 



GM Form Screens 
N=196 

Frequency Percent* 



3 
1 
49 
248 



1.5% 
.5% 
25.0% 



*Content of respondent answers was frequently coded with more than one 
code; therefore, percentages exceed 100% and frequencies exceed N's of 
387 for Direct Command Language and 196 for GM Form Screens. 



TABLE 69 



WILLINGNESS TO PARTICIPATE IN FOLLOW-UP STUDY 
N-2342 

Cumulative 

Frequency Percent Percent 

Yes 2016 86.1 86.1 

No 326 13.9 100.0 

Total 2342 



No answer: 



374 



ATTACHMENT 1 



r 

L 



INSTRUCTIONS: The purpose of this questionnaire is to find out how individuals use MEDLINE on 
the National Library of Medicine computer system, their level of satisfaction with the system, and their 
views on how it can be improved. The NLM hopes to use this information to provide better service to 
its users. Unless>othcrwise indicated, answer each question by either writing your answer in the space 
provided, or bv circling the number in front of the appropriate answer. All your answers will be available 
only to the study investigators, unless otherwise required by law. If you have any questions about this 
study, please contact Karen Wallingford at (301) 496-3261. 



SECTION I. GENERAL INFORMATION 

L What is your profession? (Circle all that apply) 

L Physician 
1 Nurse 

3. Other Health Professional 

4. Scientist 

5. Student 

6. Librarian/Information Specialist 

7. Other (specify) 

2. What is the highest educational degree you hold? 

3. What year did you receive that degree? 

4. If you are a health professional, what is your specialty? (If you are not a health 
professional, please skip this question.) 



5. What is your primary work place? (Circle only one answer) 

~T. Private solo practice 

1 Group practice 

3. College or university or medical school 

4. Hospital or clinic 

5. Private company or business 

6. Government agency 

7. No formal work place (ie, student or otherwise unaffiliated) 

8. Other (specify) 

6. Do you have a microcomputer (PC) or is one available in your work place? 

1. No 

1 Yes, type: 

7. How many people (including yourself) share the MEDLINE User ID Code you use? 
(If you are the only person who uses this code, please write T.) user(s) 



NATIONAL LIBRARY OF MEDICINE 

8600 Rockville Pike 
f\ Bethesda,MD 20894 




8. How many MEDLINE searches do you do on the NLM computer in the average month? 
(Do not include searches done for you by someone else.) 

searches 

9. How many MEDLINE searches do you have someone else do for you in the average 
month? (If you do all of your searches, enter a zero and skip to question 11.) 
searches 

10. If other people occasionally or always search MEDLINE for you: 

A. Who generally does the searches for you? (Circle only one answer) 

L Librarian/Information Specialist 

2. Student/Research Assistant 

3. Secretary/Administrative Assistant 

4. Colleague 

5. Family Member 

6. Other (specify) 

B. Under what circumstances do you prefer to have someone else search MEDLINE 
for you? (Circle all that apply) 

1. When someone else can do it as easily as I can 

2. When I don't have rime to do it myself 

3. When I need different expertise/system knowledge 

4. When I've tried a search myself and have not been satisfied with the results 

5. Other (specify) , 



C. Please rate how satisfied you generally are with searches you do yourself, and 
searches that are done for you by others. (If you never search MEDLINE yourself, 
please leave that response blank.) 

Very Not At All 

Satisfied Satisfied 

Searches done by: 

1. Yourself 1 2 3 4 5 

2. Others 1 2 3 4 5 

D. If you are generally not satisfied with MEDLINE searches done for you, (if you 
circled 4 or 5) please indicate why. (Circle all that apply) 

1. Inconvenient location 

2. Inconvenient hours 

3. Have to wait to get search done 

4. Cost 

5. Unsatisfactory results 

6. Other (specify) 



IF YOU NEVER SEARCH MEDLINE YOURSELF, DO NOT FILL OUT THE REST OF THIS 
QUESTIONNAIRE. PLEASE RETURN IT IN THE ATTACHED POSTAGE PAID ENVELOPE. 
THANK YOU FOR YOUR COOPERATION. IF YOU DO SEARCH MEDLINE YOURSELF, 
PLEASE COMPLETE THE REST OF THIS QUESTIONNAIRE. 



SECTION TT. SYSTEM USE 

11. What factors influence you to search online databases yourself, instead of having 
someone else do the search for you? (Circle all that apply, and check the single most 
influential factor.) 

Influential 



Factor Factor 

1. I am more familiar with the subject matter than a search 

intermediary 

2. I get the information faster 

3. I enjoy searching 

4. It's more cost effective than using a search intermediary 

5. No one else is available to do the search for me 

6. Other (specify) 



12. How experienced a user of online databases do you consider yourself to be? 

L Very experienced 

2. Somewhat experienced 

3. Not very experienced 

4. Not at all experienced 

13. How long have you been searching MEDLINE on your User ID code? 

years months 

14. During the time you have been searching, would you say that your use of 
MEDLINE has: 

L Increased 

2. Stayed about the same 

3. Decreased 

15. If your usage has increased or decreased, please indicate the reasons for the change. 



16. How often do cost considerations keep you from doing a MEDLINE search on the 
NLM computer? 

1. Never 

2. Rarely 

3. Occasionally 

4. Frequently 

17. How would you rate your overall satisfaction with MEDLINE on the NLM computer 



system? 

Very Not At All 

Satisfied Satisfied 

1 2 3 4 5 



SECTION TTT. MEDLINE SEARCHES 

18. When you search MEDLINE- 

A. Is it typically for (Circle only one answer) 

1. An immediate information need 

2. Staying current in your field 

3. Learning about new areas 

4. Other (specify) 

B. Are you typically interested In retrieving: (Circle only one answer) 
L Just a few relevant citations 

2. All relevant citations from a particular time period 

3. Other (specify) . 

C Do you typically retrieve: ( Circle only one answer) 
L Too few citations 

2. About the right number of citations 

3. Too many citations 

D. What percent of these citations are typically relevant to your inquiry? 
% 

19. When you search MEDLINE, do you most often search ton (Circle only one answer) 

1. an author 

2. a journal title 

3. a subject 

20. Please indicate the primary areas in which you use MEDLINE search information, 
rank ordered so that your most common use is #1, second most common is #2, etc 
Please give no more than three answers. 

Rank 
Order 

Patient Care 

Education 

Research/Testing 

_^ Management/Administration 

Regulation 

Other (specify) 

21. How often do you use the Medical Subject Headings (MeSH) terms when searching 
for specific subjects? 

1. Always 

2. Usually 

3. Occasionally 

4. Rarely 

5. Never 

22. How useful do you find the MeSH terms to be? 

Very Not At All 

Useful Useful 
1 2 3 4 5 



23. If you think the MeSH terms are generally not useful, or if you never use MeSH 
terms, please indicate why. 



24. How long does it typically take you (at the terminal or microcomputer) to search 
MEDLINE on the NLM system for citations on a particular subject? 

L Less than 5 minutes 

2. 5 to 10 minutes 

3. 10 to 15 minutes 

4. More than 15 minutes 

25. Do you feel that this is: 

L Too long 

2. A reasonable amount of time 

3. Quicker than expected 

26. Are there any types of information that would be valuable to you that you cannot 
routinely find in a citation? 

1. No (Skip to question 27) 

2. Yes... (Please circle all of the following types of information that would be 

valuable to you, and check the single most valuable type of information.) 

Most 
Valuable Valuable 



L Author address 

2. Dosage information 

3. Research design 

4. Journal section (ie. Brief Communications) 

5. Full text of article 

6. Other (specify) 



27. Which of the following features or capabilities would you most like to see added to 
the NLM system? (Circle as many as you like, and check the one feature you would most 
like to see.) 

Most 

Wanted Wanted 



L Ability to sort citations online 

2. Improved capability for selecting which citations to print ' 

3. Ability to sort citations among different databases 

4. Improved capability for searching MEDLINE Backfiles at one time 

5. Improved methods for SDI (automated monthly update search) service 

6. More non-English literature indexed 

7. More "didactic" (ie, educational/instructionaJ/teaching. etc) literature indexed 

8. Ability to specify the "adjacency" of searched Text Words 

9. Other (specify) 



SECTION TV. ACCESSING MF.Df.TNF. 

28. When you search MEDLINE, do you primarily use: 

L A microcomputer (PC) 
2. A terminal 

29. Please write in the percent of MEDLINE searches you perform with each of the 
following methods. Note that your percents should add up to 100. (If you do 
not use a method, please write "0".) 

% Direct/command language (no user-friendly front-end) 

% GRATEFUL MED, using form screens 

% GRATEFUL MED, using option 3, direct mode 

% Other user-friendly front-end package (specify) 



100% Total 

30. If you use more than one method of searching MEDLINE, under what circumstances 
do you choose one method over another? 



31. What types of problems, if any, do you have accessing the NLM computer? 
(Circle all that apply) 

1. No problems 

2. Remembering connect/disconnect protocols 

3. Busy telecommunication lines 

4. NLM computer not available 

5. Other (specify) 

32. How did you learn to search MEDLINE on the NLM computer system? (Please circle 
all the methods that you used, and check the one method that was most helpful to you in 
learning how to search MEDLINE.) 

JJsed Most 

Method Helpful 

1. Using GRATEFUL MED 

2. Using other front-end software (eg, SCI-MATE) 

3. Attended NLM-sponsored training course 

4. Attended a course as part of an academic curriculum 

5. Attended other, non-NLM sponsored training course 

6. Self-taught 

7. Learned from a co-worker 

8. Other (specify) ___ 



33. If you have attended an NLM-sponsored training course (choice 3 in question 32), 
please circle the course(s) you attended, and how satisfied you were with the 
course(s). 

Very Not At All 

Attended? Satisfied Satisfied 

1. 3-5 day Initial Training Course 1 2 3 4 5 

2. 6 hour Basics of Searching MEDLINE 1 2 3 4 5 



At some future point the NLM may conduct additional research on topics related to MEDLINE and the 
MEDLINE search system. Would you be willing to participate in a follow up study? 

L Yes (Please fill out your name, address and phone number below.) 
2. No 

Name: 

Address: 



Phone: 



If you have any additional comments that you would like to make about MEDLINE, please do so in the space 
below. We are particularly interested in knowing those aspects of MEDLINE with which you are most satisfied, 
and those aspects of MEDLINE with which you are least satisfied. Please continue on the back of this form if 
you need additional space. 

Most Satisfactory Aspects: 



Least Satisfactory Aspects: 



Please return this survey in (he enclosed postage paid envelope. 
Thank you for your time and cooperation. 



ATTACHMENT 2 



DEPARTMENTOF HEALTH & HUMAN SERVICES 



Public Health Service 



National Institutes of Health 
National Library of Medicine 
Bethesda MO 20694 



Dear Colleague: 

The National Library of Medicine's basic mission is to improve the 
dissemination of information important to the progress of medicine and 
to the public health. In support of this mission, NLM developed 
MEDLINE, the first online bibliographic database available via a 
nationwide telecommunications network, in 1971. In the first decade of 
its existence, MEDLINE' s principal users were medical librarians who 
acted as search intermediaries for practitioners and researchers needing 
to locate pertinent information in the biomedical and health care 
literature. In the past few years, however, the number of individuals 
who search MEDLINE for themselves has increased dramatically. To 
provide better service to these individual end-users, NLM needs to know 
more about them, how they use MEDLINE, their level of satisfaction with 
the current system, and their views on changes that would make the 
system more effective. 

In an effort to obtain this information, the Library is surveying all 
individuals who search MEDLINE directly on the NLM computer without the aid 
of a search intermediary. I hope you will take the approximately 15 minutes 
needed to fill out the enclosed questionnaire and return it in the postage 
paid envelope provided. While participation is voluntary, we would 
appreciate receiving your completed questionnaire by October 16. 

If you do not search MEDLINE yourself, that is, if you always use a 
search intermediary, please write across the top of the questionnaire 
that you do not search MEDLINE yourself, and return it in the postage 
paid envelope. If you have any questions, contact Karen Wallingford at 
(301) 496-3261. 

In appreciation for your time and participation, a $5.00 credit will be 
applied to your invoice for MEDLINE system use during the month of December 



The information collection in this study is authorized under Section 465 

of the Public Health Services Act. Your responses will be confidential 

and your answers will be available only to the study investigators, 

unless otherwise required by law. Survey results will be presented in 
the aggregate. 

NLM is working to provide information services which can assist you in 

your work. Please help us to make MEDLINE more useful to you. 

Thank you for your cooperation. 



1987. 




Donald A. B. Lindberg, M^ 
Director 



Enclosure 



ATTACHMENT 3 



Dear Colleague: 

Recently you were sent a questionnaire regarding your use of MEDLINE. If 
you have mailed it to us, thank you for your participation. If you have not yet 
returned your completed questionnaire, please mail it as soon as possible to: 
NLM, 8600 Rockville Pike, Bethesda MD 20894. Attn: K. Wallingford, Bldg. 
38, B1W-28. 

If you need another copy of the questionnaire, call Karen Wallingford at 
(301) 496-3261. 




DomVdA. B. Lindberg, M.D. - 
Director, National Library of Medicine 



ATTACHMENT 4 
DEPARTMENT OF HEALTH & HUMAN SERVICES 



Public Health Service 



National Institutes of Health 
National Library of Medicine 
Bethesda MO 20894 



Dear Colleague: 

Approximately four weeks ago you were sent a questionnaire that 
asked about your use of MEDLINE. The National Library of Medicine is 
very interested in your views, but has not yet received a completed 
questionnaire from you. As you may recall, the study is intended to 
assist us in making MEDLINE responsive to the information needs of 
persons like yourself who search MEDLINE directly without the aid of a 
search intermediary. Participation is voluntary, but if you have not 
yet completed and returned the questionnaire, please take a few minutes 
and complete the enclosed duplicate copy today. If you have already 
returned the questionnaire, thank you for participating. 

If you do not search MEDLINE yourself, that is, if you always use a 
* search intermediary, please write across the top of the questionnaire 
that you do not search MEDLINE yourself, and return it in the postage 
paid envelope. If you have any questions, contact Karen Wallingford at 
(301) 496-3261. 

In appreciation for your time and participation, a $5.00 credit will be 
applied to your invoice for MEDLINE system use during the month of 
December 1987. 

The information collection is authorized under Section 465 of the Public 
Health Services Act. Your responses will be confidential and your 
answers will be available only to the study investigators, unless 
otherwise required by law. Survey results will reported in the 
aggregate. 

Thank you for your cooperation. 




Donald A. B. Lindberg, M.D. 
Director 



Enclosure 



ATTACHMENT 5 

Explanation of Queries Used to Pull Respondent Sample 
and Breakout of Sample by MMS User Classification Code 

There were three queries run against the MEDLARS Management Section's 
(MMS's) USERS file in which basic information on users of the NLM 
databases is stored. In each query, we were only interested in those 
users who had an active billing code. This did not ensure that they 
actively used their code, but, at least, they had not asked for it to be 
canceled. MMS uses a set of classification codes (see attached) to 
identify the different types of users on the system. There are also a 
set of administrative codes originally developed to identify the type of 
institution e.g., federal, state, non-government/not-for-prof it , for 
prof it/commercial -- which were refined in 1986 to try to track the 
growing number of individuals requesting codes. Both sets of codes have 
undergone various revisions over the years which had to be taken into 
consideration in refining queries for pulling the respondent sample. 

The following is a breakout of the respondent sample according to the 
MMS classification codes: 



Class. // in 

Code Classification Type Sample 

100' s Direct Health Care 3112 

200' s Health Education All 

300 's Health-Related Research or Resource 602 

400 's Legislative, Regulatory, Planning Agencies - 19 

500' s Scientific or Technical Products and Services 48 

600' s General Products and Services 97 

700' s Information resource/Library 6 

800' s Other educational institutions/Personnel 18 



Total 



4313 



**** NLM MEDLARS USER TYPE CLASSIFICATION **** 
0 - No Classification Type Assigned (includes NLM in-house 

and foreign codes 
100 - DIRECT PATIENT CARE 

101 - Hospital/inpatient facility (inc. osteopathic, 

psychiatric) 

102 - Outpatient facility/HMO 

103 - Practitioner, physician (inc. osteopaths and 

M.D. ' s) 

104 - Practitioner, dentist 

105 - Practitioner, veterinarian 

106 - Practitioner, nurse 

107 - Other direct care 

108 - Students in health education 
200 - HEALTH EDUCATION 

201 - Academic Health Science Center 

202 - Medical School 

203 - Osteopathy School 

204 - Dental School 

205 - Veterinary School 

206 - Nursing School 

207 - Pharmacy School 

208 - Optometry School 

209 - Podiatry School 

210 - Chiropractic School 

211 - Public health/health administration program 

212 - Allied health program 

213 - AHEC/continuing professional education program 

214 - Other health-related education program 

215 - University/college not specified above 

216 - Prof., teacher, instructor in health education 
300 - HEALTH-RELATED RESEARCH OR RESOURCE 

301 - Medical/biomedical research institution 

302 - Scientific research (primary functions outside 

the scope of biomedicine 

303 - Medical/scientific society or association 

304 - Medical/scientific library or information 

resource 

305 - Health care consumer/patient support group 

306 - Health insurance 

307 - Other support activity 

308 - Medical/biomedical research 

400 - LEGISLATIVE, REGULATORY, PLANNING AGENCIES 

401 - Health administration/health planning/HSA 

402 - Health care/drug regulation or legislation 

403 - Scientific regulation/legislation/administration 

404 - Environment/energy/space technology/agriculture/ 

safety regulation, legislation, or 
administration 

405 - Military application 

406 - Law enforcement 

407 - General regulation/legislation/administration 



500 - SCIENTIFIC OR TECHNICAL PRODUCTS AND SERVICES 

501 - Pharmaceuticals 

502 - Health care supplies/equipment/services 

503 - Chemicals/plastics/petroleum/cosmetics 

504 - Food/agriculture 

505 - Energy/environment/space technology 

506 - Computers/electronics/engineering 

507 - Other scientific/technical product or service 
600 - GENERAL PRODUCTS AND SERVICES 

601 - (Not currently in use for classification) 

602 - Information broker 

603 - Publisher/media 

604 - Law firm/ lawyer 

605 - Insurance other than health insurance 

606 - Trade/welfare 

607 - Other 

608 - Science writer 

700 - INFORMATION RESOURCE OR LIBRARY 

701 - Information resource/ library 
800 - OTHER EDUCATIONAL INSTITUTIONS /PERSONNEL 
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